In the new environment we let Netbackup dictate which tapes are in the scratch pool and free for re-use based on the expiration policy of the jobs written to those tapes. In theory this should mean that you only ever use as many tapes as you need for a given backup job. The problem with this set-up is that although it is, in theory, more efficient, you don’t always know which tapes you will need each night and how many you have free.

Each day guys were running NBU command lines to show which tapes were in the scratch pool, but having to manually sift through them to see which tapes were currently sat in drives/libraries and of the ones held off-site, which tapes were of which format (LTO2, 3 or 4).

The following script can be scheduled to email out a list of tapes in the scratch pool, which are in what library and also groups the offsite tapes by format. It was knocked up in a hurry so isn’t as parameterised or dynamic as it should be, but the comments should help you fix it for your needs…

Here’s the script and HTML header & footer…

scratch_list.ps1

$debugpreference = "continue"

#This "output" variable runs and stores the output of the NBU command line
#Be sure that this .exe is in your path, otherwise change the below to include the full path to the executable.
$output = vmquery.exe -b -pn Scratch
$htmlfile = ".\output.txt"
$today = Get-Date

#Arrays to hold and sort info about our tapes.
#NOTE: this script is hard-coded to work for LTO2 (HCART2), LTO3 (HCART3) and LTO4 (HCART) tape drives/libraries
[array]$aTapes = @()
[array]$aHOffsite = @()
[array]$aH2Offsite = @()
[array]$aH3Offsite = @()

#NOTE: this script is hard coded to include the specific names of tape drives attached (via various media servers) to the master server
$aRobots = @{"TLD0" = 0; "TLD1" = 0; "TLD2" = 0; "TLD3" = 0; "TLD4" = 0; "TLD5" = 0; "TLD6" = 0; "TLD7" = 0; "TLD8" = 0; "TLD9" = 0; "TLD10" = 0; "TLD11" = 0; "TLD12" = 0; "TLD13" = 0; "TLD14" = 0}
$emailBody = Get-Content "header.html"
$aOffsite = 0

#Function to send mail (currently configured to send via Exchange)
function sendEmail() {
 $eServer = New-Object system.net.mail.smtpClient("exchange_server01")

 $From = "netbackup@yourcompany.com"
 $To = "nbu_reports@yourcompany.com"

 $todayDate = [string] $today.day + "/" + [string] $today.month + "/" + [string] $today.year

 $title = "Scratch tape list for $todayDate"

 $Body = $emailBody
 $eMsg = New-Object System.Net.Mail.MailMessage($From,$To,$title,$body)

 $eMsg.IsBodyHTML = $True

 $eServer.send($eMsg)    
}

#Take output of NBU command line util, and for each line pick out bits of the string
#Chuck everything in to a new array called $aTapes
foreach ($line in $output){
 if (($line.contains("HCART")) -and (-not $line.contains("A00"))) {

 $tName = ($line.substring(0, 6)).trim()
 $tType = ($line.substring(8, 6)).trim()
 $tRobot = $line.substring(23, 5).trim() #this captures a wide area with some white space that needs trimming

 $tRobot = $tRobot.replace(" ", "")

 $aTapes += ,($tName, $tRobot, $tType)
 }
}

#Go through the array we just created and then put the contents in to seperate arrays
#dependant on whether they are LTO, LTO2 or LTO3 tapes...
for ($j = 0; $j -lt $aTapes.length; $j++) {
 $tapeID = $aTapes[$j][0]
 $robot = $aTapes[$j][1]
 $type = $aTapes[$j][2]

 if ($robot.contains("-")) {
 switch ($type ) {
 "HCART" {$aHOffsite += ,($tapeID)}
 "HCART2" {$aH2Offsite += ,($tapeID)}
 "HCART3" {$aH3Offsite += ,($tapeID)}
 }

 } else {#If a library type does not contain HCART, HCART2 or HCART3 we must assume it's offsite (or in a safe))
 $temp = "TLD$robot"
 $aRobots[$temp] = $aRobots[$temp] + 1
 }
}

#Take the contents of $aRobots (offsite tapes) and put it in to a variables ready to be dumped in to our html output
foreach ($element in $aRobots) {
 $element > $htmlfile
}

#Start pipping everything out to the $emailBody variable which will make the body of our email
$emailBody += '<table><tr><td>Tape drive</td><td># scratch tapes</td></tr>'
foreach ($line in Get-Content $htmlfile) {

 if ($line.contains("TLD")) {
 $emailBody += "<tr>"

 $robot = $line.substring(0, 5)
 $count = $line.substring(31, 3)

 $emailBody += "<td>$robot</td><td>$count</td>"

 $emailBody += "</tr>"
 }
}
$emailBody += '</table><br/><br/>'

$emailBody += "<b>Offsite tapes...</b><br/>"

$emailBody += "<b>HCART</b><br>"
foreach($element in $aHOffsite) {
 $emailBody += "$element<br/>"
}
$emailBody += "<br/><b>HCART2</b><br>"
foreach($element in $aH2Offsite) {
 $emailBody += "$element<br/>"
}
$emailBody += "<br/><b>HCART3</b><br>"
foreach($element in $aH3Offsite) {
 $emailBody += "$element<br/>"
}

$emailBody += Get-Content "footer.html"

sendEmail

As per my other NBU scripts I use HTML files to store the CSS and opening HTML in order to easily format nice looking emails. See below for examples…

header.html

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
 <title>Scratch tape list</title>
 <style type="text/css">
 body {
 font-family: Calibri, Arial, Helvetica;
 color: black;
 font-size: small;
 background-color: #FFFFFF }
 p {
 font-size: small }
 .big {
 font-size: small }
 table {
 font-size: small;
 padding: 0px ;
 border-spacing: 2px ;
 empty-cells: hide ;
 border: 1px solid #AAAAAA }
 td {
 background-color: #EEEEEE;
 vertical-align: top }
 th {
 background-color: #EEEEEE;
 vertical-align: top ;
 text-align: left}
 </style>
</head>
<body>

<p>*** This email is now automatically generated ***<br></p>

footer.html

<br/>
<p>Regards,<br/><br/>
</p>
</body>
</html>

Let me know if this helps you out at all or if you can suggest any improvements…

Until now our users have demanded daily reports to be emailed out to key individuals. The format of these emails was quite basic in theory…it simply showed a list of all servers (clients) that were backed up, their status from last night (e.g. successful, failed, etc…) but then, crucially, a list of files that had been skipped.

The reason for listing each file that had failed was due to Netbackup’s poor poor (don’t get me started on how poor) feedback in the Activity Monitor and it’s built in reports. You see sometimes jobs could be marked as “Successful with warnings” if the job completed but the odd file had been skipped (if it were in use, for example). But other times, although it still listed which files had been missed if you double-clicked on each job, it showed the overall status as Successful. This meant that to report on the servers in the format that was required by our users we had to manually trawl through the 360 jobs across two Netbackup servers just to copy and paste the lines of the files that had been missed.

Needless to say it didn’t take me long to get bored of this so I started looking at a way of reproducing our reports automatically. After some time, I’ve come up with a solution – it uses VBScript to parse the logs files which I output using a few key Netbackup command-line tools.

The basic process and components to the script are as follows…

1. A VBscript calls a number of external batch scripts
2. These batch script run NBU commands and spit their output to text files
3. The VBscript then reads these text files
4. The VBscript reads a servers.txt file to determine what jobs to report on*
5. A LOT of array and string manipulation is used to catalogue which jobs have run and which have errors and if so, it collates all skipped files
6. These arrays are then parsed and dumped to either an HTML file (which can be used directly in the body of an email) or, as I’ve recently changed to script, outputs to an Excel file.

There are some nice bits to the script(s) and some really nasty bits – I welcome you to improve on the original and share it with everyone (for example, I know there is a much neater way of calling and parsing the output of batch jobs rather than the nasty wscript.sleep I have used).

* A text file (called servers.txt) is kept along side the scripts. This file is manually edited and must be kept up to date. The file is a number of lines, each one representing a client/job that we wish to report on. The format is as follows:

60, myserver, mypolicy, GroupName

The number at the start represents what days of the week the report should be generated. myserver is obviously the client that is to be reported on. mypolicy is the name of a policy that you are reporting on (since a client may me a member of multiple policies). Finally, GroupName is free text and can be used to group similar clients (that may not necessarily be grouped by the same policy.

The scripts necessary to generate a report are…

1. generateReport.vbs
   The primary script which calls the others and also performs of the processing and HTML/Excel generation
2. jobs_summary.bat
   Issues the bpdjobs – simmary command and pipes output to a text file. This provides a simple overview of number of total jobs and the number that have succeeded, failed or errored.
3. jobs_report.bat
   This is the main batch script upon which the report is generated. It is a hideous mess of space-formatted garbage and why so much processing is needed to generate a useful and nice looking report.

Clicking the scripts above will take you to a separate page with the code and more explanations of how to use it in your environment.

UPDATE: Okay, so enough of you have expressed an interest that I decided to put it up for public download. Please note that it is not currently protected via any kind of license so please use your discretion. I have included a very basic READ ME that I encourage you to go through. Post your comments here if you have any problems.

Download: Dave’s Netbackup Reporting script