Using CrashPlan to backup a network share

12 03 2010

Some time ago I started using CrashPlan with CrashPlan Central to finally start backing up my family photos and important documents off-site. It’s been working pretty well, despite my frustrations over slow upload speeds. It’s pretty much a fire and forget solution.

Recently I’ve installed FreeNAS on to an old PC to (a) start getting some stability to my storage solution, and (b) play around with ZFS and iSCSI at home. I tried OpenFiler but it pales in comparison to FreeNAS. I’ve gone against my better judgement and I’m using some of the cooler plugins including the Torrent client, Dynamic DNS and I’ve even installed SABNzbd on it. FreeNAS comes with rsync built in and even Unison which is an interesting solution to cross platform backup/synchronisation, but I’ve payed for my CrashPlan Central plan so want to make use of it!Only problem now is backing up my data!

CrashPlan doesn’t support backing up from a network share – either mapped drive or UNC path. There is a work around on the CrashPlan site which works but is a little convoluted so wanted to post a nice, quick version here.

Read on after the break… Read the rest of this entry »


Comments : 6 Comments »

Categories : Batch, Other Backups

Netbackup Report Scripting

25 02 2010

Hi everyone. I’m going to jump right in to my first post (the main reason for starting this blog in the first place) and show you the progress I’ve made with automating daily reports using our backup solution of choice, Symantec (used to be Veritas) Netbackup.

Until now our users have demanded daily reports to be emailed out to key individuals. The format of these emails was quite basic in theory…it simply showed a list of all servers (clients) that were backed up, their status from last night (e.g. successful, failed, etc…) but then, crucially, a list of files that had been skipped.

The reason for listing each file that had failed was due to Netbackup’s poor poor (don’t get me started on how poor) feedback in the Activity Monitor and it’s built in reports. You see sometimes jobs could be marked as “Successful with warnings” if the job completed but the odd file had been skipped (if it were in use, for example). But other times, although it still listed which files had been missed if you double-clicked on each job, it showed the overall status as Successful. This meant that to report on the servers in the format that was required by our users we had to manually trawl through the 360 jobs across two Netbackup servers just to copy and paste the lines of the files that had been missed.

Needless to say it didn’t take me long to get bored of this so I started looking at a way of reproducing our reports automatically. After some time, I’ve come up with a solution – it uses VBScript to parse the logs files which I output using a few key Netbackup command-line tools.

The basic process and components to the script are as follows…

  1. A VBscript calls a number of external batch scripts
  2. These batch script run NBU commands and spit their output to text files
  3. The VBscript then reads these text files
  4. The VBscript reads a servers.txt file to determine what jobs to report on*
  5. A LOT of array and string manipulation is used to catalogue which jobs have run and which have errors and if so, it collates all skipped files
  6. These arrays are then parsed and dumped to either an HTML file (which can be used directly in the body of an email) or, as I’ve recently changed to script, outputs to an Excel file.

There are some nice bits to the script(s) and some really nasty bits – I welcome you to improve on the original and share it with everyone (for example, I know there is a much neater way of calling and parsing the output of batch jobs rather than the nasty wscript.sleep I have used).

* A text file (called servers.txt) is kept along side the scripts. This file is manually edited and must be kept up to date. The file is a number of lines, each one representing a client/job that we wish to report on. The format is as follows:

60, myserver, mypolicy, GroupName

The number at the start represents what days of the week the report should be generated. myserver is obviously the client that is to be reported on. mypolicy is the name of a policy that you are reporting on (since a client may me a member of multiple policies). Finally, GroupName is free text and can be used to group similar clients (that may not necessarily be grouped by the same policy.

The scripts necessary to generate a report are…

  1. generateReport.vbs
    The primary script which calls the others and also performs of the processing and HTML/Excel generation
  2. jobs_summary.bat
    Issues the bpdjobs – simmary command and pipes output to a text file. This provides a simple overview of number of total jobs and the number that have succeeded, failed or errored.
  3. jobs_report.bat
    This is the main batch script upon which the report is generated. It is a hideous mess of space-formatted garbage and why so much processing is needed to generate a useful and nice looking report.

Clicking the scripts above will take you to a separate page with the code and more explanations of how to use it in your environment.

UPDATE: Okay, so enough of you have expressed an interest that I decided to put it up for public download. Please note that it is not currently protected via any kind of license so please use your discretion. I have included a very basic READ ME that I encourage you to go through. Post your comments here if you have any problems.

Download: Dave’s Netbackup Reporting script


Comments : 12 Comments »

Categories : Batch, Netbackup, Powershell, Scripting, VBscript

jobs_report.bat

25 02 2010

This batch is called by the main generateReport.vbs script. It used built-in Netbackup command line utilities to output the contents of the NBU database report section to a CSV whith the VBscript then imports.

1:  REM *** THIS BATCH FILE IS CALLED AUTOMATICALLY BY generateReports.bat ***
2:  REM *** Run bpdbjobs for todays report...
3:  @bpdbjobs -report -gdm > logs\report.csv
4:  REM *** Run again to add to master report...
5:  @bpdbjobs -report -gdm >> backups\reports_master.csv

Comments : Leave a Comment »

Categories : Batch, Scripting, Uncategorized

job_summary.bat

25 02 2010

This simple batch simply generates a summary of recent jobs (total, number failed, etc…)

1:  REM *** THIS BATCH FILE IS CALLED AUTOMATICALLY BY generateReports.vbs ***
2:  @bpdbjobs -summary -L > logs\summary.log

Comments : Leave a Comment »

Categories : Batch, Scripting, Uncategorized