| |
Contents of this Section
- Overview
- Compute nodes
- Interconnect
- Storage
- Technical summary
Overview
Lawrencium is a 198-node, 1584 processor Linux cluster equipped with
a high performance, low latency Infiniband interconnect that is suitable
for providing an stable, high performing resource for running wide
diversity of scientific applications. Currently, it is coupled
with a BlueArc high performance NFS storage server that provides
a total of 48TB home and 9TB scratch shared filesystem space to users.
Lawrencium is currently ranked at number 500 on the November 2008
Top500 list of the world's fastest supercomputers.
Compute nodes
Each node is a Dell poweredge 1950 equipped with two Xeon Intel quad-core 64-bit Harpertown processors (8 cores in all) on a single board configured as an SMP unit. The core frequency is 2.66GHz and supports 4 floating-point operations per clock period with a peak performance of 10.64 GFLOPS/core or 85 GFLOPS/node. Each node contains 16GB of memory. The memory subsystem has dual channel 1333 MHz Front Side Bus connecting to 667 MHz Fully Buffered DIMMS. Both processors share access to the memory controllers in the memory controller hub (HCM or North Bridge).
Interconnect
A high performance, low latency interconnect utilizing DDR Infiniband technology provides a bandwidth of 20Gb/s. The topology is a fat tree configured with a 3:1 blocking factor. Specifically, the thirty-six nodes of each frame are connected to 2 Cisco 7000D 24-port DDR Infiniband switches (leafs). (eighteen nodes per leaf switch). From each leaf switch, the remaining 6 ports are used to uplink to the core fabric consisting of six Cisco 7000D 24-port DDR Infiniband switches. A total of 6 frames are connected in this fashion to the core fabric to support the 198 compute nodes. The core fabric has capacity to support up to 432 nodes.
Storage
The cluster is equipped with high performance NFS storage from BlueArc with 9 TB of 10K RPM Fibre Channel disks for scratch and 48TB of 7200RPM SATA disk storage for home
directories and projects.
Technical summary
| Component |
Description |
| System Name |
Lawrencium |
| Login Host |
lrc-login.lbl.gov |
| Operating System |
CentOS 5 x86_64 (based on RedHat Linux) |
| Number of nodes (cores) |
198 (1584) |
| Total Aggregate Memory |
3.1 TB |
| Performance |
16.8 TFlops (Peak); 12.6 TFlops (Linpack) |
| Processor |
Intel 2.66Ghz Xeon Quad core 64-bit Harpertown |
| Network interconnect |
Double Data Rate Infiniband (20 gb/s) |
| Resource Manager |
PBS Torque |
| Job Scheduler |
Moab |
| Disk (Storage) |
Bluearc NFS (9TB Scratch & 48TB Home)
Data Direct Networks Lustre (150TB) |
| User Environment Management |
Modules |
| Serial Compilers |
Intel ver 10.1: Fortran 90/95, C, C++
GNU ver 4.1.2: Fortran 90/95, C, C++ |
| Parallel Compilers |
OpenMPI ver 1.2.7 (built with above Intel compilers) |
|
|
|
|
|
| |
|
|
| |
|
|
| 
