First things first: this post (and subsequent posts) are built off the backs of many great NetApp employees, especially the PS organization and the cmode-ses mailing list. In particular I’d like to thank Luke Mun, Doug Moore, Barry Spencer and Pavel Sobiezczuk.
This post will describe how to convert a system that’s currently running Data ONTAP 8.1 7-mode into 8.1 cluster-mode. If you’re using 8.0 7-mode, I strongly recommend upgrading to 8.1 7-mode before the conversion: there’ll be a couple less steps involved. I’ll assume you’re using a Cisco Nexus 55xx switch as the private cluster interconnect. While this isn’t required per se, it is the only option that’s properly supported.
The Cisco switch setup is actually quite simple. You can get the full, NetApp-approved configuration from the NetApp support site here. Here are the relevant global options for the switch(es):
feature lacp errdisable recovery interval 30 errdisable recovery cause pause-rate-limit policy-map type network-qos cluster class type network-qos class-default mtu 9216 system qos service-policy type network-qos cluster spanning-tree port type edge default port-channel load-balance ethernet source-dest-port
You can see that we’re not doing a whole lot: enabling LACP (the Link Aggregation Contro Protocol), building a basic QoS policy and setting some error parameters. Now, here are the relevant port options for cluster nodes:
interface Ethernet1/1 description Cluster Node 1 no lldp transmit no lldp receive spanning-tree port type edge spanning-tree bpduguard enable
Here, you can see we’re disabling LLDP (the Link Layer Discovery Protocol) and configuring the switch ports for spanning-tree protocol. Last but not least, if you’re connecting two Cisco Nexus switches together, here are the relevant options for the inter-switch link:
interface Ethernet1/13 description Inter-Cluster Switch ISL Port 13 (Port-channel) no lldp transmit no lldp receive switchport mode trunk channel-group 1 mode active
Not much to see here, either: disable LLDP, define STP (this time as trunk nodes) and note that the link is part of a channel group (i.e. an aggregated link). Once you’ve done the Cisco configuration, you’re off to the races!
From here, we boot the Filer. Once you hit the LOADER prompt, hit Ctrl-C to interrupt the boot process. Enter these options to configure the Filer to boot cluster-mode now and in the future:
set-defaults setenv bootarg.init.boot_clustered true setenv bootarg.init.usebootp false setenv bootarg.bsdportname e0a
After you’ve set these options, boot Data ONTAP by entering boot_ontap. Once Data ONTAP starts booting, hit Ctrl-C again to get to the installation options. You’ll see a menu like this:
(1) Normal Boot. (2) Boot without /etc/rc. (3) Change password. (4) Clean configuration and initialize all disks. (5) Maintenance mode boot. (6) Update flash from backup config. (7) Install new software first. (8) Reboot node.
Select option (4) Clean configuration and initialize all disks. This will re-zero the disks, which can take an hour or two. After re-zeroing, your Filer will reboot automatically and proceed to the setup wizard.
And that’s enough for now!