RADIAL - UPRM - Proposed Worker Node Specs

Head Node

$18,874

• 2 Xeon Silver 4310, 2.1GHz, 12core, 2U, 12-bay, $7,750 Dell PowerEdge R750   Dell PowerEdge R750
• Memory
  • 2 x 32GB, DDR4-3200 RDIMM, 2 x $158 = $366 SK-HYNIX HMA84GR7DJR4N-XN
• Drives
  • 4 x 960TB 2.5" SSDs (OS RAID1), 3 x $770 = $2,310 Samsung PM863 (3 drives for the array and one cold spare on a shelf)
  • 10 x 16TB 3.5" SAS HDDs (DATA RAID6), 9 x $330 = $2,970 https://www.newegg.com/seagate-exos-x18-st16000nm004j-16tb/p/1Z4-002P-022Y4?Item=1Z4-002P-022Y4 (9 drives for the array and one cold spare on a shelf)
• Drive Caddies
  • 12 x 3.5" drive caddies for PowerEdge R750, WorkDone 12-pack $198 https://www.amazon.com/WorkDone-4-Pack-Compatible-Installation-Screwdriver/dp/B08VF2Y7XG/?th=1

Compute Node options

The test node purchased for NRAO was a Compute GPU Node, common and this is what we are recommending for all the other clusters.

Compute GPU Node, common

(760W Dell recommends 208V to get the full capacity of the Power Supply)

I unknowingly bought power supplies for the NRAO test system with C20 recepticals instead of c14 recepticals..

$17,416

•  2 Xeon Silver 4309Y, 2.89GHz, 8core, 2U, 8-bay,  2400W, $9,369  Dell PowerEdge R750    Dell PowerEdge R750
• Memory
  • 8 x 32GB, DDR4-3200, RDIMM, 8 x $158 = $1,264 https://memory.net/product/hma84gr7djr4n-xn-sk-hynix-1x-32gb-ddr4-3200-rdimm-pc4-25600r-dual-rank-x4-module/
• NVMe Drive
  • Samsung PM1733 MZWLJ7T6HALA, 7.68TB, U.2, NVMe, $1,583 https://www.cdw.com/product/samsung-pm1733-mzwlj7t6hala-ssd-7.68-tb-pcie-4.0-x4-nvme/6254372
• Drive Caddies (only need to order 2 12-packs total)
  
  • 2.5" drive caddy for PowerEdge R640 (should also work with R650), WorkDone 12-pack, $200 https://www.amazon.com/WorkDone-2-Pack-Hard-Drive-Caddy/dp/B07Q4T91B3/
• GPUs
  • 2 x NVIDIA RTX A5000, 230W, 2 x $2,600 = $5,200 https://www.amazon.com/NVIDIA-Quadro-RTX-A5000-Graphics/dp/B098R6RKXL

Compute CPU Node, Silver

(300W) $11,010

• 2 Xeon Silver 4309Y, 2.89GHz, 8core, 1U, 10-bay,  $6,899 Dell PowerEdge R650
•  Memory
  • 16 x 32GB, DDR4-3200, RDIMM, 16 x $158 = $2,528 https://memory.net/product/hma84gr7djr4n-xn-sk-hynix-1x-32gb-ddr4-3200-rdimm-pc4-25600r-dual-rank-x4-module/
• NVMe drive

  • Samsung PM1733 MZWLJ7T6HALA, 7.68TB, U.2, NVMe, $1,583 https://www.cdw.com/product/samsung-pm1733-mzwlj7t6hala-ssd-7.68-tb-pcie-4.0-x4-nvme/6254372

• Drive Caddies
  
  • 2.5" drive caddy for PowerEdge R640 (should also work with R650), WorkDone 12-pack (only need to order 2 or 3), $200 https://www.amazon.com/WorkDone-2-Pack-Hard-Drive-Caddy/dp/B07Q4T91B3/

Compute CPU Node, Gold

(400W) $14,663

• 2 Xeon Gold 6334, 3.6GHz, 8core, 1U, 10-bay,  $10,552 Dell PowerEdge R650
•  Memory
  • 16 x 32GB, DDR4-3200, RDIMM, 16 x $158 = $2,528 https://memory.net/product/hma84gr7djr4n-xn-sk-hynix-1x-32gb-ddr4-3200-rdimm-pc4-25600r-dual-rank-x4-module/
• NVMe drive

  • Samsung PM1733 MZWLJ7T6HALA, 7.68TB, U.2, NVMe, $1,583 https://www.cdw.com/product/samsung-pm1733-mzwlj7t6hala-ssd-7.68-tb-pcie-4.0-x4-nvme/6254372

• Drive Caddies
  
  • 2.5" drive caddy for PowerEdge R640 (should also work with R650), WorkDone 12-pack (only need to order 2 or 3), $200 https://www.amazon.com/WorkDone-2-Pack-Hard-Drive-Caddy/dp/B07Q4T91B3/

Power Strips

• 120V (Rack supports 2 0U power strips on each side, or four total.  There isn't going to be that much cabling, just Ethernet, so there is certainly room for 2 and maybe up to 4 0U power strips.  If you need more than 4 power strips, the extras will have to be 2U.)
  
  • APC AP8932, 0U, 120V, 2.8kW, NEMA L5-30P input, NEMA 5-20R output, $1,450 https://www.apc.com/shop/us/en/products/APC-Rack-PDU-2G-switched-0U-30A-100V-to-120V-24-NEMA-5-20R-sockets/P-AP8932
    
  • APC AP7902B, 2U, 120V, 2.8kW, NEMA L5-30P input, NEMA 5-20R output, $1,375 https://www.apc.com/shop/us/en/products/Rack-PDU-Switched-2U-30A-120V-16-5-20/P-AP7902B
    
• 208V
  • APC APDU9965, 8.6kW, 0U, Input is NEMA L21-30P 3phase, Outputs are 21x C13/C15 and 3x C19/C21, $2,100 https://www.apc.com/shop/us/en/products/APC-Rack-PDU-9000-switched-0U-8-6kW-208V-21-C13-and-C15-3-C19-and-C21-sockets/P-APDU9965
  • APC APDU9967, 17kW, 0U, Input is IEC60309 60A 3P+PE 3Phase, Outputs are 42x C13/C15 and 6x C19/C21, $3,775 https://www.apc.com/shop/us/en/products/APC-Rack-PDU-9000-switched-0U-17-3kW-208V-42-C13-and-C15-6-C19-and-C21-sockets/P-APDU9967

Switch

• Cisco Catalyst C9300X-48TX
  

  • 48 Data, 48x 10G Multigigabit

  • 100M, 1G, 2.5G, 5G, or 10 Gbps
  • Switching capacity 2,000 Gbps
  • Nine optional Modular Uplinks 100G/40G/25G/10G/1G
  • Redundant Power Supply 715 W
  • $12,000

Environment Monitor

(only need one)

• APC AP9335TH, Temperature and Humidity Sensor, length is 3.9m, $190 https://www.apc.com/shop/us/en/products/APC-Temperature-Humidity-Sensor/P-AP9335TH

Rackmount KVM

• StarTech RKCONS1901, Rackmount KVM console, 1U, $990 https://www.cdw.com/product/startech.com-rackmount-kvm-console-1u-19-lcd-vga-kvm-drawer-w-cables-usb/5103418?pfm=srh

Rack Cabinet

• APC NetShelter SX AR3100, $1,875 https://www.apc.com/shop/us/en/products/APC-NetShelter-SX-Server-Rack-Enclosure-42U-Black-1991H-x-600W-x-1070D-mm/P-AR3100 

Power Cables (Top power)

• Nodes + spare
  • 3-foot, C13/C14, Y-splitter (wide), qty 17
• Switch + spare
  • 6-foot, C13/C14, qty 2
• KVM + spare
  • 2-foot, C13/C14, qty 2

Power Cables (Bottom power)

• Nodes + spare
  • 4-foot, C13/C14, Y-splitter (wide), qty 17
• Switch + spare
  • 4-foot, C13/C14, qty 2
• KVM + spare
  • 3-foot, C13/C14, qty 2

Ethernet Cables

• 10-foot, Cat-6a, qty 7
• 7-foot, Cat-6a, qty 8
• 5-foot, Cat-6a, qty 4

Blank Panels

• One 4U, SKU#: 102-1825 from https://www.racksolutions.com/filler-panels.html
• Two 2U, SKU#: 102-1823 from https://www.racksolutions.com/filler-panels.html

2U Rack Drawer

• One 2U SKU: KH-1922-3-100-02 2U drawer https://www.rackmountsolutions.net/kendall-howard-1922-3-100-02-2u-rack-mountable-drawer/

UPS (Optional)

(just for switch and head node)

• 120V
  • APC Smart-UPS SMT1500RM2UC 120V $1,050
• 208V
  • APC Smart-UPS X SMX2200R2HVNC 208V, $3,125

Software

Since we are planning to use all free and open-source software (RockyLinux and HTCondor) there isn't a need for software costs.