David Warburton » Netapp

Using Powershell to report on Netapp filer snapshots

DavidWarburton — Tue, 02 Mar 2010 09:21:17 +0000

We had a few issues recently where SnapManager for Exchange and SnapDrive were failing to communicate properly which resulted in a weeks worth of missed snapshots before anyone realised what was going on.

I wrote this script to report on the status of snapshots. The filer name is passed as a variable and a list of volume names is stored as a variable at the top of the script. The output is nicely formatted email.

The script uses SSH to connect to the filer (see my SSH guide for Netapp filers)
For each vol listed in the array, the list of snapshots is returned
Any line with “__daily” in is processed – all others are ignored (we only care about our nightly verified snapshots which are all named __daily)
The snapshot name contains the date/time it was taken – this is read and if the date is within 24 hours of now a success is return (since reports are run the day after the verified job). If the date is over 24 hours a failure is returned.
A nicely formated HTML email is sent out which looks a little something like this…

Please see below for today’s backup report.

VOL Name	Status	Snapshot name
Vol_Exc02_DB	Success	0% ( 0%) 0% ( 0%) Mar 02 01:44 exchsnap__exchange02_03-01-2010_23.00.14__daily (busy,backup[0],dump)
Vol_Exc02_LOG	Success	0% ( 0%) 0% ( 0%) Mar 02 04:38 eloginfo__exchange02_03-01-2010_23.00.14__daily
Vol_Exc02_DB1	Success	0% ( 0%) 0% ( 0%) Mar 01 23:00 exchsnap__exchange02_03-01-2010_23.00.14__daily
Vol_Exc02_LOG1	Success	1% ( 0%) 0% ( 0%) Mar 02 01:27 eloginfo__exchange02_03-01-2010_23.00.14__daily
Vol_Exc03_DB	Success	1% ( 1%) 0% ( 0%) Mar 02 00:50 exchsnap__exchange03_03-01-2010_23.00.13__daily
Vol_Exc03_LOG	Success	0% ( 0%) 0% ( 0%) Mar 02 02:06 eloginfo__exchange03_03-01-2010_23.00.13__daily
Vol_Exc03_DB1	FAILURE	1% ( 1%) 0% ( 0%) Feb 26 23:00 exchsnap__exchange03_03-01-2010_23.00.13__daily
Vol_Exc03_LOG1	FAILURE	0% ( 0%) 0% ( 0%) Feb 26 00:32 eloginfo__exchange03_03-01-2010_23.00.13__daily

See the script and HTML files after the jump…

Don’t forget to change the name of your mail server, your sending and recipient email addresses and, most importantly, the location of plink.exe and your DSA keys used to establish the SSH connection (although you could easily modify this script to use RSH instead).

snapReport.ps1 script

# ==============================================================================================
#
# NAME:     Netapp filer snapshot reports
#
# AUTHOR:     David Warburton
# DATE:     12/03/2009
#
# PARAMETERS:    1. $filer
#        Specifies the name of the remote filer to connect to.
#        example: snapReport.ps1 myfiler01
#
# ==============================================================================================
$debugPreference = "continue"
$errorActionPreference = "stop"
$filer = $Args[0]
$htmlFile = "D:\Logs\Exchange Snapshot Report\exchange_Snapshot.html"
$volumes = ("Vol_Exc02_DB", "Vol_Exc02_LOG", "Vol_Exc02_DB1", "Vol_Exc02_LOG1", "Vol_Exc03_DB", "Vol_Exc03_LOG", "Vol_Exc03_DB1", "Vol_Exc03_LOG1")
$header = get-content "header.html"
$header > $htmlFile
$today = get-date
function sendEmail() {
 $eServer = new-object system.net.mail.smtpClient("mailserver.yourdomain.com")
 $From = "youraddress@yourcompany.com"
 $To = "backup_reports@yourcompany.com"
 if ($checkType -eq "weekly") {
 $title = "Weekly "
 } else {
 $title = "Daily "
 }
 $todayDate = [string] $today.day + "/" + [string] $today.month + "/" + [string] $today.year
 $title += "Exchange snapshot Report for $todayDate"
 $Body = get-content $htmlFile
 $eMsg = new-object System.Net.Mail.MailMessage($from,$to,$Title,$body)
 $eMsg.IsBodyHTML = $True
 #$eMsg.cc.add($cc)
 $eServer.send($eMsg)
}
function checkSnap($volName) {
 $output = D:\Utils\PuTTY\plink.exe svc-ssh@$filer -i D:\Utils\PuTTY\id_dsa.ppk snap list $volName
 $validsnap = $False
 $snapname = ""
 foreach ($line in $output){
 if ($line.contains("__daily")) {
 $sMonth = $line.substring(62, 2)
 $sDay = $line.substring(65, 2)
 $sYear = $line.substring(68, 4)
 $sDate = [datetime] "$sMonth $sDay $sYear"
 if (($sDate.date).addDays(1) -eq $today.date) {
 $validsnap = $True
 $snapname = $line
 }
 }
 if ($validsnap) {
 return $snapname
 }
 }
 return $False
}
foreach ($vol in $volumes) {
 $line = "$vol"
 $status = checkSnap $vol
 if ($status -eq $false) {
 $line += "FAILED"
 } else {
 $line += "Success"
 }
 $line += "$status"
 $line >> $htmlFile
}
$footer = Get-Content "footer.html"
$footer >> $htmlFile
sendEmail

Here are the HTML bits for the header and footer of the email…

header.html

Backup Report

*** This email is now automatically generated ***

Please see below for today's backup report.

NOTE:
- Exchange and SQL snapshots are managed by SnapManager and are alerted independantly to this report.
- Not all snapshots are shown in these reports. Hourly snapshots are currently not reported upon.

footer.html

VOL Name	Status	Number of Snapshots	Last snapshot taken

Regards.

NTFS permissions lost when copying from a snapshot

DavidWarburton — Mon, 01 Mar 2010 10:04:24 +0000

Something I just remembered when performing a restore for a user.

When copying files from a CIFS snapshot using the hidden .snapshot directory NTFS permissions are lost. Instead they are inherited from the folder you are copying to.

To maintain permissions, use NDMPCOPY specifying the snapshot name in the source (first) parameter…

ndmpcopy "filer1:/vol/vol_A/qtree_A/.snapshot/nightly.1/MyDocs" "filer1:/vol/vol_A/qtree_A/MyDocs"

You can enclose the path in quotes if the path has spaces in.

Netapp Rapid Cloning Utility v3.0

DavidWarburton — Fri, 26 Feb 2010 16:02:01 +0000

Netapp have released their new version of their Rapid Cloning Utility – a vCenter plugin which allows you to provision and new datastores and clone hosts (including VMware View 4 VDI’s) with ease right inside of the Virtual Infrastructure Client. vCenter 4 is needed but it is compatible with ESX 3.5 and 4.

The great thing is that all the storage processing is offloaded from vCenter and is performed entirely on the filers. I’ve not had a chance to play with the RCU yet but this just looks utterly awesome! Check out this preview blog post and video from Netapp:

http://blogs.netapp.com/virtualstorageguy/2009/12/preview-rapid-cloning-utility-30-vcenter-plug-in.html

If you don’t have time to sit and read, then at least check out the video. I dare you not to be impressed!

Do any other storage vendors have similar tools?

Source: http://www.ntapgeek.com/2010/02/netapp-updates-rcu-for-vmware-vsphere-4.html

Scripting Netapp filers

DavidWarburton — Fri, 26 Feb 2010 14:05:50 +0000

Scripting the Netapp filers can be accomplished by using either RSH or SSH. Both of these options must be enabled on the filer(s) in question using.

In either case scripts are not processed on the filers themselves. They are scheduled and run on Windows/Linux systems using their native languages (e.g. Vbscript or Powershell). These scripts simply call native OnTAP commands against a filer which then returns data in the form of text. This is important to note as although Powershell is a useful language to use OnTAP will not return native PS objects which means you are limited to text/string based manipulation.

RSH

This method is the least secure but is the quickest and easiest to set up. Users need not necessarily be filer administrators. Although there is some degree of security in the sense that access is restricted to a set user account and accessible only via a set IP address, data is not encrypted.

RSH is not natively supported in Windows 2008 [from the command line] as it is in Windows 2003.

To enable RSH on the filer use the following command:

options rsh.enable on

You must also specify user accounts and IP addresses via the FilerView web interface.

SSH

Windows 2008 does not natively support SSH either so this is accomplished using plink.

To enable SSH on the filer use the following commands:

secureadmin setup
secureadmin enable ssh

Configuring SSH

First create a local user account on the filer using the following command:
```
 useradmin user add svc-ssh -g Administrators
```
1. the passwords are kept in the Password spreadsheet
2. The user must be a member of Administrators to be granted the Login_ssh permission on the filer
Using Powershell (whilst running with your Admin Account) change directory to the hidden share on the filer:
```
 cd \\filer\c$\etc\sshd\
```
Create a folder with the same name as your account created in step 1 and then another folder to hold the SSH keys:
```
 md svc-ssh
 md svc-ssh\.ssh
```
On the script server lunch PUTTYGEN.EXE and under the “Key” menu change the type to “SSH-2 DSA key”. Leave everything else on its default setting.
Click on the Generate button and follow the prompt
Enter the same password from step 1 in the “Key passphrase” and “Confirm passphrase” areas (they do not HAVE to be the same as step 1 but are being kept so for simplicity)
When complete, click on Save public key. Save it to the PuTTY folder with the name:
id_dsa.pub
Save the private key to the same place with a similar name:
id_dsa.ppk
Copy the contents of the “Public key for pasting into OpenSSH authorised_key file” area and paste it in to a new text document. You should use Wordpad to do this as Notepad does not handle the line breaks properly. Save this document as authorized_keys.
You are now finished. You can test connection using PuTTY by selecting your private key. The first time you connect you will be prompted to accept the hosts key. You will still be prompted for a username but you should not need to enter a password as this is what the SSH keys are for.

For scripting purposes we use plink.exe to issue a command. For example:

.\plink.exe svc-ssh@filer -i "id_dsa.ppk" df -s

Scripting using plink

The basic use of plink.exe is as follows:

.\plink.exe svc-ssh@filer -i "id_dsa.ppk" df -s

Two methods can be used to script the filers depending on what you’re trying to achieve:

Reporting	Grabbing the contents of a single command and piping it to a Powershell variable. This allows us to manipulate the string (or lines of strings) that are returned. Objects are not returned so true Powershell scripting (i.e. Object Orientated) can not be done but PS gives us some very useful string maniplation functions.
Maintenance	Issuing multiple commands to the filer to perform repetitive/scheduled actions. For example, deleting, renaming and taking new snapshots.

NOTE: when calling plink.exe in Powershell the full or abbreviated path must be specified. So use either a full path, for example, D:\Utils\PuTTY\plink.exe or just .\plink.exe

Reporting

From a Powershell command prompt simply pipe the output of an SSH command in to a variable for later manipulation:

$output = .\plink.exe svc-ssh@filer -i “id_dsa.ppk” df -s

Maintenance

If multiple command need to be issued – for example, you need to delete a snapshot, rename the previous 5 snapshots, and then take a new snapshot – you should issue them all via the one SSH connection.

Commands can be concatenated using a semi-colon. Just be sure to enclose everything in quotes. For example:

 .\plink.exe svc-ssh@filer -i "id_dsa.ppk" "snap delete Vol_A hourly.6; snap rename Vol_A hourly.5 hourly.4; snap rename Vol_A hourly.4 hourly.3; snap rename Vol_A hourly.3 hourly.2; snap rename Vol_A hourly.2 hourly.1; snap create Vol_A hourly.0"

Well that’s it. It’s a pretty brief post – if you have any questions please feel free to post them in the comments…