Description
The Dell PowerEdge C6100 Family – Which one is which?
This list was first compiled in the forums but it is very important as there are several options available. Let’s break down the 6xxx cloud computing family from Dell.
Dell C6100 XS23-SB versus XS23-TY3
The Dell C6100 XS23-SB was the first generation part that can be found easily today. The XS23-SB had four nodes but with dual Core 2 generation LGA 771 Xeons. The processor of choice was the 50w TDP Intel Xeon L5420. Today these are generally found for $100-200less than the XS23-TY3 version. We are still confirming whether these have compatible drive trays. Power supplies tended to be much smaller than later versions. Unless there is a burning reason to get the XS23-SB – skip it.[pullquote_right]Bottom line: Get the Dell C6100 XS23-TY3[/pullquote_right]
The focus of the piece today is the Dell C6100 XS23-TY3. The most common configuration seems to have been dual L5520‘s with 24GB DDR3 per node (6x 4GB DDR3 DIMMs.) These can accept six DIMMs per socket and with two DIMMs per node and four nodes that means 48 DIMM slots. As mentioned previously the Dell C6100 XS23-TY3 came in both 2.5″ and 3.5″ drive options, with the 3.5″ drives being much more widely used. Here is an overview of what these look like.
Only the top two nodes are visible but there are two identical nodes in-between. One can see four fans in the midplane. Quite amazing that there is one fan allocated for each dual socket node.
Other Dell PowerEdge C6000 Family Members
- The Dell C6105 is a 4- node dual Opteron 4000 series version of the C6100.
- Dell C6145 – Only two nodes here, but of quad Opteron 6000 series chips so still 8 CPUs/ node. The major benefit of this is that one can take advantage of high density with shared fans and power supplies.
- Dell C6220 – I look at this as rev2 for the C6100 series as noted above. These can handle higher TDP CPUs, have hot swap fans and etc. They utilize Intel Xeon E5-2600 based nodes supporting higher than and have 1200w power supplies.
These are either too new or too rare to be found easily second hand. The Dell PowerEdge C6220 looks great but is harder to come by unless you are buying thousands of units directly from Dell.
Why is the Dell C6100 XS23-TY3 Exciting? They are INEXPENSIVE!
At this point we have three Dell C6100 XS23-TY3’s at Fiberhub in Las Vegas, our colocation site and one inbound for the lab. Although we have access to higher-end and newer hardware, these were chosen because they are amazingly inexpensive.
I purchased the disk less unit for $1000 + 65 Shipping. Here is a link to the “deal” thread. Here is an ebay Dell XS23-TY3 search where several diskless units have popped up for $975 + 45 shipping. The common configuration at this price is:
One 1.1kW power supply in a 3.5″ drive chassis with rackmount rails and four nodes. Each node has:
- Dual Intel Xeon L5520 (4C/8T 2.26GHz) 60w TDP
- 24GB DDR3 Memory (6x 4GB RDIMMs in each node.
- 1x 3.5″ drive sled, wired for up to 3x drives/ node
- 2x Intel Gigabit LAN and 1x IPMI LAN port
Generally speaking at around $1,000 these units do not come with redundant power supplies nor full 12-drive compliments. One may need to add as necessary for those costs.
Quick math tells us that is $267/ node including rackmount rails. Performance wise these are faster in heavily threaded workloads than the Intel Xeon E3-1290 v2 but have the advantage of accepting up to 96GB of inexpensive DDR3 RDIMMs in each node compared to the Intel Xeon E3-1200 series at 32GB per node. The XS23-TY3 also uses triple channel memory versus dual. At $267/ node that’s about the price of an Intel Xeon E3-1240 V2before RAM, motherboard chassis and etc.
Why they are so inexpensive… thanks big cloud players!
How can the prices be this low? Where do the Dell C6100 XS23-TY3 units on ebay come from? Rumor has it they are mostly they are off-lease equipment that large cloud providers purchased through Dell Financial Services. The Dell C6100 XS23-TY3 3.5″ option with 8x L5520’s and 96GB are the most common versions being dumped on the secondary market. Allegedly they are being sold for $700-800 each, but to buy the off lease equipment one must purchase in 500 unit quantities. It also seems like some came with LSI SAS controllers and Mellanox dual port Infiniband QDR ConnetX-2 controllers. Usually these are not in the inexpensive versions.
Surely they use a lot of power and are loud right?
With redundant 1.1kW power supplies and a very dense configuration, one might expect these to suck down power and be very loud. Since most come with L5520’s rated at 60W TDP and the maximum per CPU is 95w TDP, using lower power CPUs does a lot to lower the overall power consumption (35w * 8 CPUs = 280w of head room right there.) It should also be noted that in addition to the power button in the rear of the chassis, the front chassis ears also have power and reset buttons.
Using a sample test configuration with 8x L5520, 24x 4GB DDR3 RDIMM, redundant 1100w PSUs:
Idle: 174w at 66dba
Folding@Home GROMACS 100% CPU load on all 8 CPUs: 489w at 77.4dba (power saving enabled)
Overall not bad. At maximum load that is around 1A @ 120v per node. One forum member, PigLover is actively pursuing a project to bring these to no more than 50dba for home-lab use. See the forums for his Taming the 6100 project.
Servicing the Dell C6100
After installing three of these in a rack in the DC, they are absolutely great to work on. Swapping motherboards is done by unscrewing a single screw then pushing the latch while pulling units out.
Power supplies can also be swapped out from the rear of the unit. The front is entirely hot swap as well:
As one can see from the colocation picture, there is a chassis not hooked up. These are so inexpensive one can buy a bare bones (no CPU/ RAM or sell the CPUs and RAM) to stock a cold spare in the event that anything that is not hot-swap fails. Swapping all drives four motherboards and two power supplies takes under 5 minutes.
Conclusion and where to go for more information
Overall, these are very easy to work on and for the price, they make a very compelling case. There is an official Dell C6100 XS23-TY3 forum thread with a few thousand views already. PigLover’s Taming the 6100 project is another one to keep an eye on. If you are looking to purchase one, use this ebay search. Be careful. There are sellers selling two node units, single CPU per node units, way overpriced barebones and filled units and etc. They are worth a look but remember, most sellers are purchasing these in lots of several hundred because the Xeon L5520 CPUs sell for $45+ after transaction fees ($360 total) and the RAM for $15 ea x 24 ($360 total) meaning selling anything else is profit. Two key takeaways here. First, there are many of these available at low prices. Second, these may be the best value cloud server or cluster nodes out there at the moment.
