NetApp port and cabling

A FAS3240 (single-chassis HA) for example has two SAS, two GBIC slots, two GBICs with FCP modules, five RJ45 and a USB – and that’s without using the two PCI expansion slots or the chassis I/O expansion.

Here I’ll break down the ports you see on the back of a FAS3240 and describe the three “new” management methods e0M, SP and ACP.

The documentation that ships with the controller has a good legend to help decipher the icons printed by each connection:

And a picture of the chassis, just in case you are not standing next to it.

I’ll go left-to-right

7: SAS cables for your external SAS shelves. 0a, 0b. If you ordered your filer with SAS shelves it should have shipped with cables. Note that you can multi-path shelves with SAS just like FCP.

8: HA cables. c0a, c0b. If you buy a multi-chassis filer (ie you want a FAS3240 with the expansion module you’ll get two physical chassis) or want to do metro-cluster. Note also – these are 10G Ethernet ports but they have OS-set IP addresses and cannot be used for anything but HA.

6: FCP. 0c, 0d. Used for connecting to Fibre Channel switches, shelves or tape drives (for NDMP backups)

3: e0a and e0b. Two 1GB Ethernet adapters for network access and traditional management. iSCSI, CIFS, NFS, http will all use these ports.

4: Unlocked management. e0M / SP combo port. This is the successor to BMC/RLM out-of-band management port. It adds a twist in that port e0M on the filer shares this port (there is an internal hub). You can configure just the SP port (sp setup) and use it for out-of-band management, you can also configure e0M from the filer which will use this port for in-band management – Filerview, System Manager etc. Note that you are required to have e0M on a separate IP network than your other ethernet ports, and it is required to be on the same network as the Service Processor. It is possible data (iSCSI, NFS, CIFS) traffic will go out this port as a last resort – but it won’t have it’s own default gateway. This needs to be on a dedicated management-only network. Do not configure e0M unless you intend to use it – the filer will default management traffic to it if it is configured.

5: Locked management. e0P. Used for Alternate Control Path (ACP) Out-Of-Band (OOB) management of SAS shelves More and More

1/2: Serial port and RJ45-to-DB9 adapter. Used to manage the filer via console. Note you’ll need one of these if you have a modern system to connect the old-school serial cable up to your modern USB port. Don’t forget putty also.

Not labeled: USB. Currently described as “unused”

Always configure SP. Don’t configure e0M unless SP is connected to a dedicated management network. Always connect the ACP cables (hey, you paid for it!).
Always multipath SAS and FCP when you have the ports available.
Always take the lifeboat with the motor.

---

Many of the specifics of the DS2246 are encoded in the product name:

  DSxxxx: Disk shelf
  DS2xxx: 2U
  DSx24x: 24 drives
  DSxxx6: 6Gbps SAS (aggregate 24Gb/sec)

---

http://www.sysadmintutorials.com/tutorials/netapp/netapp-disk-shelf-cabling-examples/

Netapp Disk Shelf Cabling Examples

This page is dedicated to a few examples I’ve put together in Visio to highlight the correct SAS and ACP cabling combinations for Netapp disk shelf cabling.

Two types of controllers and two types of disk shelves are used in these examples. They are:

• Netapp FAS2040
• Netapp FAS3240
• Disk Shelf DS4243
• Disk Shelf DS2246

A few notes before we get started:

• Netapp FAS2040 use single-path HA
• SAS Cables can be SAS copper or SAS optical or a mix.  EXCEPT Shelf-to-Shelf connections in a stack must be all SAS copper cables or all SAS optical cables. If the Shelf-to-Shelf cable is SAS copper then the controller to shelf cable must also be SAS copper. If the Shelf-to-Shelf cable is SAS optical then the controller to shelf cable must also be SAS optical.
• SAS optical cables connected to disk shelves require a disk shelf firmware that supports SAS optical cables
• The total end-to-end path, from controller to the last shelf, cannot exceed 510 meters.
• Square Ports are always cabled to circles ports, and circle ports are always cabled to square ports. Never cable square ports to square ports or circle ports to circle ports.
• The illustrations below show the Green Cable as the SAS cable and the Red Cable as the ACP cable.

Let’s dive into some examples.

FAS2040 – 1 Netapp Disk Shelf

This example shows a Netapp FAS2040 with 2 controllers connected to a single DS4243 or DS2246 Netapp disk shelf.

FAS2040 – 2 Netapp Disk Shelves

This example follows on from the previous example by adding an additional disk shelf for a total of 2 disk shelves.

FAS2040 – 3 Netapp Disk Shelves

This example follows on from the previous example by adding another additional disk shelf for a total of 3 disk shelves.

FAS3240 – 1 Netapp Disk Shelf

This example shows 2 Netapp FAS3240’s connected to a single DS4243 Netapp disk shelf.

FAS3240 – 2 Netapp Disk Shelves

This example follows on from the previous example by adding an additional disk shelf for a total of 2 disk shelves.

FAS3240 – 3 Netapp Disk Shelves

This example follows on from the previous example by adding an additional disk shelf for a total of 3 disk shelves.

FAS3240 – 6 Netapp Disk Shelves in 2 Seperate Stacks

In this example we use 3 Netapp DS4243 disk shelves in Stack 1 and 3 Netapp DS2246 disk shelves in Stack 2. The requirement for this configuration is 4 SAS ports. I have added the 4-Port SAS expansion card Netapp X2065A into each controller.

If you have any technical questions about this tutorial or any other tutorials on this site, please open a new thread in the forums and the community will be able to help you out.