Information - NetApp DS4243 / DS4246 - Hardware Raid - Windows Server

reference
netapp
experience
windows
10gig
hardware-raid
#1

Hey Everyone!

Its your resident Gohan472 here!

I am here to talk to you about the NetApp DiskShelf (DS4243 / DS4246)

I came into a small fortune of NetApps, 9x to be exact. I have 4x in my 42U Rack currently. (I plan to grow into them)

However, I do NOT use UnRaid. (Just personal preference currently)
Your probably thinking, “You dont use Unraid, what an uncultured swine, what is it then… Storage Spaces in windows??? PSSSHHH”

No, I do not use Storage Spaces either. :wink:
BUT I do use Windows!

I am one of the small percentage of people that still use a Hardware Raid Card ( LSI 9286cv-8e )

This bad boy is equipped to handle up to 255 disks, SSD Caching, 1GB onboard cache, Battery Backup Unit, and is just a beast in the sheets :smirk:

High Performance, Minimal Configuration, Parity, etc, etc, and it can all be configured from Windows!

Thats right, using the Mega Raid Storage Manager or MegaCLI


The Raid Card does have a limitation in that you can not use it as a JBOD aka IT Mode card. You MUST use it with Raid Striping. This isnt an issue for me.

Anyways, back to the NetApps

The NetApp DS4243 & DS4246 require a “Mini SAS (SFF-8088) - QSFP+ cable”, 2x to be exact if you want to maximize performance and use both IOM3 or IOM6 modules in one DiskShelf.

These cables can be pretty pricey about $70ea +tax and shipping, but I found that you only need 2x max since you can “daisy chain” the rest of the NetApps off the primary NetApp using the regular QSFP+ <-> QSFP+ cables that are native for the NetApps

The NetApps themselves are detected as a Disk Enclosure and natively interface with the 9286 Raid Card.

They also report data such as Voltage, Fan Condition, Fan Speed, Drive Temperature, and Disk Shelf Temperature, PSU detection.
(None of these values can be changed manually, but after further research, I have found that the Fan Speed Profiles baked into the NetApps are triggered when the sensors detect higher heat, thus if you have a low ambient room temperature, you will have lower fan speeds)

Performance

The NetApps do have a slow single disk Read / Write speeds (1x Disk RAID 0)

1x 2TB RAID 0

BUT once you start adding drives together in Raid 0 that speed bumps up significantly

2x 2TB RAID 0

4x 2TB RAID 0

image


I found that a big factor in maxing out the performance of this setup has to deal with the READ and the WRITE from both the source and the destination

I was reading from 1x Single drive and writing to 2x Drives in a Raid 0

Since the read speed of the Source disk was maxing out at 65MB/s to 150MB/s the read speed was slowing down the entire transfer.

Cost of Equipment

LSI 9286cv-8e Raid Card ($30 to $100 on Ebay)
2x Mini-SAS (SFF-8087) to QSFP+ cables roughly $70/ea
NetApps (I cant really factor in this cost for myself, however they do run like $100-200 depending on if you can even acquire one)


Network

(WS1) Windows Server #1 {10Gb SFP+ network card}
(4x 10TB Raid 0) LSI 9265-8i internal drives

(WS2) Windows Server #2 {10Gb SFP+ network card}
(4x 2TB Raid 0) LSI 9286cv-8e {Netapps}

WS1 and WS2 are both connected to a 16x SFP+ Port 10Gb Mikrotik Switch via SFP+ cables with 10Gb Network Cards

I have found that you can only hit a max of 6.7* Gbps from WS1 to WS2 due to a Read speed limitation of a 4x Disk Raid 0

Pics of the Setup


Front of the Rack

| Top to Bottom |
7PESH2 Server Build
Ryzen 5 2600 Server Build
Empty 4U Rosewill
4x Netapp DS4246/DS4243

Rear of the Rack


2x NetApps Connected

Conclusion

I am certain that performance could be further improved if I used more of the Raid Card features such as the SSD Caching, and maybe tried using a few of the other Raid Types.

3 Likes
#2

:rofl:

Love it. Super interesting build, not something many people are doing here! One day…

Pics of the rack? :wink:

1 Like
#3

Hey Chip!

Yeah, I added some pics of the rack to the Post!

I only have 2x NetApps connected currently. Mostly just to test them out.

1 Like
#4

NICE SETUP!!!

I JUST got a DS4246 this week. I am however going go be running a couple of Unraid boxes.

This leads me to ask… HOW MANY servers can a DS4246 attach to with its 2 controllers?

1 Like
#5

Hey @Sekrit

Congrats on getting a DS4246! They are pretty hard to come by nowadays.

From what I have recently discovered. You can NOT use the top IOM/Controller (12 bays) with server A, and the bottom IOM/Controller (12 bays) with server B.
From what I experienced. (And I need to perform a bit more testing). There is more going on under-the-hood of the netapp than we expect. I believe it has something to do with Multipath links and such.

What you could do though. Is setup 1x Unraid box with the NetApp (max of 24 Disks in unraid? iirc), and then “share” out that data/shares via a 1Gb/1Gb or 10Gb/10Gb network to the other Unraid server, that would be an option. You could rclone mount it, etc.
Supposedly there was an option to share a NetApp with 2x Netapp FS controllers. BUT i am not sure how that would work with multiple traditional RAID controllers/ IT Mode LSI adapters / NetApp HBA.
Reddit here I come ! :smiley:

As for unraid. Its very difficult to have crazy high performance due to the single disks access. (Being limited to the read/write speed of a single drive, ie, ~150MB/s for HDDs)
Unless its ALL Flash/SSD based. (single disks read/write access on SATA SSD is ~600MB/s)

1 Like
#6

VERY interesting. I truly appreciate it. I am not SUPER concerned with throughput. I’ll mostly be running to learn how to set up servers and create websites and other stuff here at home. Perhaps in the future once I learn more, i’ll worry about switching over to IOM12 and pushing SAS SSDs… But, in the meantime, I am mostly trying to figure out the best way to get the most throughput with my current 10G switch, cards, and gear. :smiley:

However, learning all that you guys are teaching me about possibilites are really covering a lot for me on system and network configuration to max out saturation where possible… And I thank you.

Kamehameha, Brother.

1 Like