Navigation

Search

CADRE

Computation and Data Resource Exchange
data-services-icon computation-icon visualization-icon storage-icon education-icon Biomedical-icon Arts-and-humanities-icon Discuss Location Icon

Rivanna Research Computing Cluster

General Specifications

Physical Attributes 

377 physical servers and 32 network switches deployed across 10 racks.

Computational Capacity Heterogeneous set of 6,360 logical cores, with 6,172 cores dedicated for interactive or job-scheduled computational research. Resources dedicated for interactive or job-scheduled computational research have a combined single-precision computational capacity of about 232,754,000 FLOPS
External Network Connections to UVA core-network switches provide the cluster with a theoretical inbound+outbound maximum network-IO throughput rate of about 2,400 MB/sec (most clients connecting wirelessly or by means of a gigEnetwork connection will experience inbound+outbound transfer rates of about 40-80 MB/sec).
Internal Network

A non-blocking CLOS-3 Infinband network has a point-to-point inbound+outbound maximum network-IO throughput rate of about 5,600 MB/sec, the Nortel Baystack point-to-point maximum network-IO throughput rate is about 120 MB/sec for up to 4,800 MB/s worth of evenly distributed network activity across the entire Nortel Baystack.

Memory Capacity

A total of 41.8 terabytes of RAM, with an additional total of 17 gigabytes of L1+L2 cache spread across all interactive or job-scheduled computational nodes.

Storage Options
  • Lustre storage as /scratch
  • Value storage volumes
  • Enterprise storage volumes
  • Swift containers

Distinguishing Features

  • 4 large-memory compute-nodes with 1,024 GB of DDR-4 RAM installed per server
  • 4,800 core homogenous set of computational resources interconnected by a low latency FDR Infiniband network
  • 56-Gbps Access to the Lustre storage (/scratch)
  • 2 NVIDIA K20 GPUs
  • Largest concentration of managed computing capacity at UVA

 

Cluster Status and Monitoring

Rivanna has Ganglia monitoring enabled and is accessible to researchers with access to the system. If you have access to Rivanna and wish to see Rivanna's Ganglia metrics please:

  1. Create a SSH tunnel between your laptop/workstation and Rivanna for port 80. For example, on a MacBook Pro you would run 'sudo su -', then 'ssh -L 8080:udc-ba30-11:80 <your eservices ID>@interactive.hpc.virginia.edu'
  2. Access this page

You may alternatively access the metrics from a web-browser running in a Rivanna FastX session, by going to: http://udc-ba30-11/ganglia (it won't work if you try it from here).

 

Detailed Interactive & Compute Node Specifications

Qty Processor Family Base Micro architecture Cores Per Node GB RAM Per Node MHz Processor Speed Memory Speed Internal Network (gbps) UVA Network (gpbs) Queue Assignment(s)
4 Intel Ivy Bridge EP Sandy Bridge 20 64 2,500 DDR3-1866 56 10 (direct) none (interactive nodes)
240 Intel Ivy Bridge EP Sandy Bridge 20 128 2,500 DDR3-1866 56 10 (routed) serial and parallel
2 Intel Sandy Bridge-EP Sandy Bridge 16 256 2,600 DDR3-1600 40 10 (routed) gpu
4 Intel Haswell-EP Sandy Bridge 16 1,024 2,600 DDR4-1866 56 10 (routed) largemem
11 Intel Sandy Bridge-EP Sandy Bridge 16 128 2,700 DDR3-1600 1 1 (routed) economy
4 Intel Westmere-EP Nehalem 8 48 2,400 DDR3-1066 1 1 (routed) economy
22 Intel Westmere-EP Nehalem 12 96 2,530 DDR3-1333 1 1 (routed) economy
7 Intel Westmere-EP Nehalem 12 96 2,670 DDR3-1333 1 1 (routed) economy
58 Gainestown Nehalem 8 48 2,670 DDR3-1333 1 1 (routed) economy
11 AMD Magny-Cours

Bulldozer

16 16 2,000 DDR3-1333 1 1 (routed) economy