Clustered Ontap basics and terminology

I know I have just dove into this, but I realize I have not discussed some basics regarding clustered ontap.  This is certainly not meant to be a detailed list, because for that you should be reading NetApp’s documentation and TRs.  However, for the cliff notes version… you have come to the right place.

In Vmware terms, think of clustered ontap as “vsphere”.  It is what you would call, everything.

clustershell–this is you talking to the hypervisor

nodeshell – this is you talking to a physical controller

systemshell – this is the underlying low-level OS system that runs under Data Ontap

cluster – collective term for all of your nodes

node – a physical controller

lif – logical interface, for maximum flexibility, you will want to have 1 lif per volume per protocol (iscsi, nfs, cifs, fc)

vserver – think of this as a virtual machine running in the cluster.  1 common question around this is “where exactly does it exist?”.  It exists anywhere in which that vserver owns disks and/or has a lif interface.

I’m sure some other terms will come to me, I will revise this list as necessary!

 

Clustered Ontap….more 8.2RC1 hints and a minor surprise

Continuing the idea from the last post, we’re updating the other node with an image that is already downloaded by doing the following:

kitt::system node image> show
Is Is Install
Node Image Default Current Version Date
——– ——- ——- ——- ————————- ——————-
kitt-01
image1 false false 8.2RC1 4/16/2013 04:37:16
image2 true true 8.1.2 1/12/2013 07:54:56

 kitt::system node image> update -node kitt-01 -package file:///mroot/etc/software/82RC1_q_image.tgz

Software update started on node kitt-01. Updating image1 package: file:///mroot/etc/software/82RC1_q_image.tgz current image: image2
Listing package contents.
Decompressing package contents.
Invoking script (validation phase).
INSTALL running in check only mode
Mode of operation is UPDATE
Current image is image2
Alternate image is image1
Package MD5 checksums pass
Versions are compatible
Available space on boot device is 747 MB
Required space on boot device is 290 MB
Kernel binary matches install machine type
Invoking script (install phase). This may take up to 30 minutes.
Mode of operation is UPDATE
Current image is image2
Alternate image is image1
Package MD5 checksums pass
Versions are compatible
Available space on boot device is 747 MB
Required space on boot device is 290 MB
Kernel binary matches install machine type
Getting ready to install image
Syncing device…
Extracting to /cfcard/x86_64/freebsd/image1…
x BUILD
x CHECKSUM
x COMPAT.TXT
x INSTALL
x VERSION
x cap.xml
x fw.tgz
x kernel
x metadata.xml
x perl.tgz
x platform.ko
x platfs.img
x rootfs.img

(some output omitted)

kitt::system node image> system image modify -node kitt-01 -image image1 -isdefault true

kitt::system node image> show
Is Is Install
Node Image Default Current Version Date
——– ——- ——- ——- ————————- ——————-
kitt-01
image1 true false 8.2RC1 4/17/2013 01:48:08
image2 false true 8.1.2 1/12/2013 07:54:56
kitt-02
image1 true true 8.1.2 12/3/2012 02:39:30
image2 false false 8.2RC1 4/16/2013 04:42:36
4 entries were displayed.
Now I will reboot kitt-01. I do not care about NDU so I did not migrate any lifs and I disabled SFO:

 

kitt::system node image> cluster ha show
High Availability Configured: true

kitt::system node image> storage failover show
Takeover
Node Partner Possible State
————– ————– ——– ————————————-
kitt-01 kitt-02 false Connected to kitt-02, Takeover is
not possible: Storage failover is
disabled
kitt-02 kitt-01 false Connected to kitt-01, Takeover is
not possible: Storage failover is
disabled
2 entries were displayed.

kitt::system node image> reboot -node kitt-01
(system node reboot)

Warning: Are you sure you want to reboot the node? {y|n}: y

login: Waiting for PIDS: 618.
Waiting for PIDS: 595.
Setting default boot image to image1… done.
.
Uptime: 50m48s
System rebooting…

Phoenix TrustedCore(tm) Server
Copyright 1985-2005 Phoenix Technologies Ltd. All Rights Reserved
Portions Copyright (c) 2005-2009 NetApp All Rights Reserved
BIOS Version: 1.10.0
CPU= AMD Opteron(tm) Processor 852 X 4
Testing RAM.
512MB RAM tested
32768MB RAM installed
Fixed Disk 0: NACF1GBJU-A11
Boot Loader version 1.9
Copyright (C) 2000-2003 Broadcom Corporation.
Portions Copyright (C) 2002-2009 NetApp

CPU Type: AMD Opteron(tm) Processor 852
Starting AUTOBOOT press Ctrl-C to abort…
Loading x86_64/freebsd/image1/kernel:…..0x100000/9519208 0xa14068/3141424 Entry at 0x8016d950
Loading x86_64/freebsd/image1/platform.ko:.0xd13000/682672 0xe7f898/629592 0xdb9ac0/40872 0xf193f0/43104 0xdc3a68/92946 0xdda57a/69080 0xdeb360/146528 0xf23c50/170928 0xe0efc0/16 0xe0efd0/2136 0xf4d800/6408 0xe0f880/0 0xe0f880/320 0xf4f108/960 0xe0f9c0/1744 0xf4f4c8/5232 0xe10090/1528 0xf50938/4584 0xe10688/184 0xf51b20/552 0xe10740/448 0xe48560/13562 0xe7f79b/253 0xe4ba60/122544 0xe69910/89739
Starting program at 0x8016d950
NetApp Data ONTAP 8.2RC1 Cluster-Mode
*************************************************
This platform is not supported in this release.
The system will now halt.
*************************************************

This is what I was eluding to yesterday. Of course, those of us in IT never RTFM but in this case it’d be a good idea it seems 🙂

Sure enough, in the 8.2RC1 release notes , page 44, we find this:

Unsupported systems in clustered Data ONTAP
Some storage systems are not supported in the clustered Data ONTAP 8.2 release family.
The following storage systems are not supported in Data ONTAP 8.2:
• 6070
• 6030
• 3040
• 2040
(thanks to the NetApp web site for this information)

Clustered ontap…8.2RC1 is here!

NetApp has posted 8.2RC1 which as a lot of new interesting features, one of which I am very eager to try, QoS for storage!  You can limit iOPs and such by volume.

But first, we are going to look at the upgrade steps.  Page 33 of the 8.2RC1 Documentation lists the 3 different states the software can be within your cluster:

1.  Download but not install

2.  Install a previously downloaded image

3.  Download & install in 1 step.

As we look at our current images:

kitt::system node image> show
Is Is Install
Node Image Default Current Version Date
——– ——- ——- ——- ————————- ——————-
kitt-01
image1 false false 8.0.2P4 9/17/2012 04:31:04
image2 true true 8.1.2 1/12/2013 07:54:56
kitt-02
image1 true true 8.1.2 12/3/2012 02:39:30
image2 false false 8.0.2P4D7 11/18/2012 04:00:00

We are in an interesting, but education state with a different version of the backup image than each other.

Now we will download + install the image to both nodes:

 

kitt::system node image> update -node -package http://10.3.3.44/DOT/82RC1_q_image.tgz -replace-package true

Error: Missing value for -node.

kitt::system node image> update -node * -package http://10.3.3.44/DOT/82RC1_q_image.tgz -replace-package true
Software update started on node kitt-01. Updating image1 package: http://10.3.3.44/DOT/82RC1_q_image.tgz current image: image2
Downloading package. This may take up to 10 minutes.
96% downloaded
Download complete.
Listing package contents.
Decompressing package contents.
Invoking script (validation phase).
INSTALL running in check only mode
Mode of operation is UPDATE
Current image is image2
Alternate image is image1
Package MD5 checksums pass
Versions are compatible
Available space on boot device is 753 MB
Required space on boot device is 290 MB
Kernel binary matches install machine type
Invoking script (install phase). This may take up to 30 minutes.
Mode of operation is UPDATE
Current image is image2
Alternate image is image1
Package MD5 checksums pass
Versions are compatible
Available space on boot device is 753 MB
Required space on boot device is 290 MB
Kernel binary matches install machine type
Getting ready to install image
Syncing device…
Extracting to /cfcard/x86_64/freebsd/image1…
x BUILD
x CHECKSUM
x COMPAT.TXT
x INSTALL
x VERSION
x cap.xml
x fw.tgz
x kernel
x metadata.xml
x perl.tgz
x platform.ko
x platfs.img
Installed MD5 checksums pass
Installing diagnostic and firmware files
Firmware MD5 checksums pass
Installation complete. image1 updated on node kitt-01.

Software update started on node kitt-02. Updating image2 package: http://10.3.3.44/DOT/82RC1_q_image.tgz current image: image1
Downloading package. This may take up to 10 minutes.
96% downloaded
Download complete.
Listing package contents.
Decompressing package contents.
Invoking script (validation phase).
INSTALL running in check only mode
Mode of operation is UPDATE
Current image is image1
Alternate image is image2
Package MD5 checksums pass
Versions are compatible
Available space on boot device is 753 MB
Required space on boot device is 290 MB
Kernel binary matches install machine type
Invoking script (install phase). This may take up to 30 minutes.
Mode of operation is UPDATE
Current image is image1
Alternate image is image2
Package MD5 checksums pass
Versions are compatible
Available space on boot device is 753 MB
Required space on boot device is 290 MB
Kernel binary matches install machine type
Getting ready to install image
Syncing device…
Extracting to /cfcard/x86_64/freebsd/image2…
x BUILD
x CHECKSUM
x COMPAT.TXT
x INSTALL
x VERSION
x cap.xml
x fw.tgz
x kernel
x metadata.xml
x perl.tgz
x platform.ko
x platfs.img
x rootfs.img
Installed MD5 checksums pass
Installing diagnostic and firmware files
Firmware MD5 checksums pass
Installation complete. image2 updated on node kitt-02.

2 entries were acted on.

kitt::system node image>

Image installed, now we verify:

kitt::system node image> system node image show-update-progress -node *
There is no update/install in progress
Status of most recent operation:
Run Status: Exited
Exit Status: Success
Phase: Run Script
Exit Message: Installation complete. image1 updated on node kitt-01.
There is no update/install in progress
Status of most recent operation:
Run Status: Exited
Exit Status: Success
Phase: Run Script
Exit Message: Installation complete. image2 updated on node kitt-02.
2 entries were acted on.

kitt::system node image> system node image show
Is Is Install
Node Image Default Current Version Date
——– ——- ——- ——- ————————- ——————-
kitt-01
image1 false false 8.2RC1 4/16/2013 04:37:16
image2 true true 8.1.2 1/12/2013 07:54:56
kitt-02
image1 true true 8.1.2 12/3/2012 02:39:30
image2 false false 8.2RC1 4/16/2013 04:42:36
4 entries were displayed.

 Next we are supposed to check out snapmirror relationships and current jobs:

kitt::system node image> job show
Owning
Job ID Name Vserver Node State
—— ——————– ———- ————– ———-
1 Certificate Expiry Check
kitt – Queued
Description: Certificate Expiry Check
2 CLUSTER BACKUP AUTO 8hour
kitt – Queued
Description: Cluster Backup Job
3 CLUSTER BACKUP AUTO daily
kitt – Queued
Description: Cluster Backup Job
4 CLUSTER BACKUP AUTO weekly
kitt – Queued
Description: Cluster Backup Job
5 Vol Reaper kitt – Queued
Description: Vol Reaper Job
5 entries were displayed.

kitt::system node image> job delete *
5 entries were acted on.

kitt::system node image> job show
This table is currently empty.

 These queued jobs are expected due to the nature of this being a lab that is spun up and spun down on demand.

kitt::system node image> storage failover show -auto-giveback false -instance

Node: kitt-01
Partner Name: kitt-02
Node NVRAM ID: 118045084
Partner NVRAM ID: 118049824
Takeover Enabled: true
Takeover Possible: true
Reason Takeover not Possible: –
Interconnect Up: true
Interconnect Links: VIA Interconnect is up (link 0 down, link 1 up)
Interconnect Type: Infiniband ( Mellanox Arbel )
State: connected
Partner State: Up
Time Until Takeover: –
Reason Takeover not Possible by Partner: –
Auto Giveback Enabled: false
Auto-Abort Operations Enabled: false
Check Partner Enabled: true
Takeover Detection Time (secs): 15
Takeover on Panic Enabled: true
Takeover on Reboot Enabled: true
Delay Before Auto Giveback (secs): 600
Hardware Assist Enabled: true
Partner’s Hwassist IP: 10.3.3.102
Partner’s Hwassist Port: 4444
Hwassist Health Check Interval (secs): 180
Hwassist Retry Count: 2
Hwassist Status: active
Time Until Auto Giveback (secs): –
Local Mailbox Disks: 7a.00.0, 7a.00.1
Partner Mailbox Disks: 5c.12.0, 5c.12.1
Missing Disks on Local Node: None
Missing Disks on Partner Node: None
Auto Giveback After Takeover On Panic: true

 Some steps have been omitted, refer to the upgrade guide for more detail.

Here is what it looked like rebooting 1 of the nodes:

kitt::system node image> reboot -node kitt-01
(system node reboot)

Warning: Are you sure you want to reboot the node? {y|n}: y

login: Waiting for PIDS: 618.
Waiting for PIDS: 595.
.
Uptime: 2h35m50s
System rebooting…

Phoenix TrustedCore(tm) Server
Copyright 1985-2005 Phoenix Technologies Ltd. All Rights Reserved
Portions Copyright (c) 2005-2009 NetApp All Rights Reserved
BIOS Version: 1.9.0
CPU= AMD Opteron(tm) Processor 852 X 4
Testing RAM.
512MB RAM tested
32768MB RAM installed
Fixed Disk 0: NACF1GBJU-A11
Boot Loader version 1.8
Copyright (C) 2000-2003 Broadcom Corporation.
Portions Copyright (C) 2002-2009 NetApp

CPU Type: AMD Opteron(tm) Processor 852

Note: The system will install firmware version 1.10.0 in 5 seconds.
Type Ctrl-C now to skip firmware installation.

Updating firmware now.

Warning: Do not power off the system until the update is complete.

New BIOS Version: 1.10.0
New Loader Version: 1.9
Saving Primary Image to Secondary
Updating Secondary Boot Flash
Programming .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+ done. 2031616 bytes written
Updating Primary Boot Flash
Programming .+.+.+.+.+.+.+.+.+.+.+.+.+.+ done. 917504 bytes written

Firmware update is complete.
System rebooting…

So far so good, but we are about to find out a minor surprise…..

 

Clustered ontap…nfs export

In the last post we talked about a vserver and an nfs export.

The rules are different in Clustered Ontap vs 7mode.  As it was mentioned you no longer use /etc/exports, but inside define export rules and attach them to vservers.

kitt::vserver export-policy rule*> create -vserver vsNFS1 -policyname policy1 -protocol nfs -rorule any -rwrule any -clientmatch 10.3.3.0/24

kitt::vserver export-policy rule*> show
Policy Rule Access Client RO
Vserver Name Index Protocol Match Rule
———— ————— —— ——– ——————— ———
vsNFS1 default 1 any 0.0.0.0/0 any
vsNFS1 policy1 1 nfs 10.3.3.0/24 any
2 entries were displayed.

Policies are applied on a per volume basis, therefore, lets look at our volume:

kitt::vserver export-policy*> volume show -volume vsNFS1_vol_data1 -instance

Vserver Name: vsNFS1
Volume Name: vsNFS1_vol_data1
Aggregate Name: aggr1_data
Volume Size: 500GB
Name Ordinal: base
Volume Data Set ID: 1026
Volume Master Data Set ID: 2147484674
Volume State: online
Volume Type: RW
Volume Style: flex
Is Cluster-Mode Volume: true
Is Constituent Volume: false
Export Policy: default

(Output truncated)

Now we will modify this so it will use the policy we created earlier, “policy1” and verify:

kitt::vserver export-policy*> volume modify -volume vsNFS1_vol_data1 -policy policy1

kitt::vserver export-policy*> volume show -volume vsNFS1_vol_data1 -instance

Vserver Name: vsNFS1
Volume Name: vsNFS1_vol_data1
Aggregate Name: aggr1_data
Volume Size: 500GB
Name Ordinal: base
Volume Data Set ID: 1026
Volume Master Data Set ID: 2147484674
Volume State: online
Volume Type: RW
Volume Style: flex
Is Cluster-Mode Volume: true
Is Constituent Volume: false
Export Policy: policy1

Mount and verify:

10.3.3.131:/vsNFS1_vol1_data1 on /home/ric/testvol type nfs (rw,vers=3,addr=10.3.3.131)

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/sda1 15480816 13613068 1081368 93% /

10.3.3.131:/vsNFS1_vol1_data1

498073600 512 498073088 1% /home/ric/testvol

Finally, we verified we could read and write data.