Lawrencium Configuration

Contents of this Section

Overview

Lawrencium is Berkeley Lab's institutional computational resource for Berkeley Lab PIs and their collaborators. It consists of 3 cluster pools, LR1, LR2 and LR3, that are each equipped with a high performance, low latency Infiniband interconnect that is suitable for providing an stable, high performing resource for running wide diversity of scientific applications. Currently, they are all coupled with a BlueArc high performance NFS storage server that provides a home directory storage and a Data Direct Networks 300TB Lustre parallel filesystem that provides high performance scratch space to users. Lawrencium LR1 was ranked at number 500 on the November 2008 Top500 list of the world's fastest supercomputers.

Compute Nodes (LR1)

LR1 is the first pool of compute for the Lawrencium cluster.consisting of 223 compute nodes. Each node is a Dell poweredge 1950 equipped with two Xeon Intel quad-core 64-bit Harpertown processors (8 cores in all) on a single board configured as an SMP unit. The core frequency is 2.66GHz and supports 4 floating-point operations per clock period with a peak performance of 10.64 GFLOPS/core or 85 GFLOPS/node. Each node contains 16GB of memory. The memory subsystem has dual channel 1333 MHz Front Side Bus connecting to 667 MHz Fully Buffered DIMMS. Both processors share access to the memory controllers in the memory controller hub (HCM or North Bridge).

Compute Nodes (LR2)

LR2 is an additional pool of 170 compute nodes intended to augment LR1 to meet the increased computational demand from LBNL researchers. Nodes are a combination of IBM iDataPlex dx360M2, HP SL390, and Dell C6100 servers each equipped with two Xeon Intel hex-core 64-bit Westmere processors (12 cores per node) on a single board configured as an SMP unit. The core frequency is 2.66GHz and supports 4 floating-point operations per clock period with a peak performance of 10.64 GFLOPS/core or 85 GFLOPS/node. Each node contains 24GB of NUMA memory connected via a triple QuickPath Interconnect (QPI) channels. LR2 has a peak performance rating of 20TF, but provides better computational efficiency making it much faster than LR1 even though the peak rating is similar.

Compute Nodes (LR3)

LR3 is a very recent addition to the Lawrencium condo cluster consisting of 108 16-core compute nodes connected with a Mellanox FDR Infiniband fabric . Each node is a Dell poweredge C6220 server blade equipped with two Xeon Intel 8-core Sandybridge processors (16 cores in all) on a single board configured as an SMP unit. The core frequency is 2.6GHz and supports 8 floating-point operations per clock period with a peak performance of 20.8 GFLOPS/core or 322GFLOPS/node. Each node contains 64GB 1600Mhz memory

Storage

The cluster is equipped with high performance NFS storage from BlueArc with 48TB of 7200RPM SATA disk storage for home directory use and a Data Direct Networks S2A9900 providing 300TB of Lustre parallel filesystem for high performance scratch space.

Technical summary

Component	Description
System Name	Lawrencium
Login Host	lrc-login.lbl.gov
Operating System	Scientific Linux 6 (based on RedHat Linux)
Number of nodes (cores)	524 (5376)
Total Aggregate Memory	15 TB
Aggregate Performance	79 TFlops (Peak)
Processor	Intel 2.66Ghz Harpertown, Westmere, Sandybridge
Interconnect	Infiniband
Resource Manager	PBS Torque
Job Scheduler	Moab
Disk (Storage)	Bluearc NFS (9TB Scratch & 48TB Home) Data Direct Networks Lustre Parallel Filesystem (300TB)
User Environment Management	Modules
Serial Compilers	Intel Fortran 90/95, C, C++ GNU Fortran 90/95, C, C++
Parallel Compilers	OpenMPI (built with above Intel compilers)