Install and optimise the performance of the University’s Linux cluster InfiniBand network in time to complete the required LINPACK benchmark tests and meet the deadline for the November, 2006 Top500 List.
Solution
585 QLogic InfiniPath® QLE7140 PCI Express host channel adapters interconnect through a SilverStrorm (SilverStorm Technologies™ acquired November, 2006 by QLogic Corporation) multi-protocol switched fabric to 585 Dell PowerEdge 1955 dual-core 3.0 GHz Xeon processor servers. The resulting nine 65-node computational unit clusters is capable of operating 27 TFlops.
Result
Supercomputing performance without a supercomputing price tag, helping Cambridge score the #2 spot in the UK, #7 in Europe, and #20 overall in the Top500 List of the world’s most powerful supercomputers.
Supercomputer Hits the Top 20
Ranked 20th in the November 2006 Top500 List, the high performance supercomputing cluster at Cambridge University is one of the most powerful supercomputers in the world. Test results by Top500.org demonstrate over 18 TFlops of supercomputing power using the LINPACK benchmark study.
http://www.top500.org/list/2006/11/100
The Cambridge system, nicknamed “Darwin”, is a Linux-based HPC cluster containing nine computational units (CU). Each CU is a pair of racks housing 65 Dell PowerEdge 1955 compute nodes which all connect to a core communications rack.
The InfiniBand interconnect consists of 585 QLogic InfiniPath QLE7140 PCI Express host channel adapters connected to nine QLogic SilverStorm 9024 InfiniBand switches at the edge and one core QLogic SilverStorm 9080 Multiprotocol fabric director all supporting Full Bisectional Bandwidth (FBB).
On a day-to-day basis, the Cambridge supercomputer is capable of processing over 27 trillion floating point operations per second (TFlops), which provides the computing horsepower to simultaneously analyse anything from complex molecular structures, to cosmology and geophysics.
Cambridge Studies QLogic
At this level, bottlenecks in the cluster interconnect simply do not make the grade. That’s why Cambridge selected QLogic InfiniPath InfiniBand Host Channel Adapters (HCAs) and a QLogic SilverStorm InfiniBand switched fabric. Known for their industry-leading performance, QLogic InfiniPath HCAs
offer the lowest latency, the highest message rate and highest effective bandwidth of any cluster interconnect available.
QLogic SilverStorm multi-protocol switches and directors provide the low-latency, high-bandwidth fabric through which the InfiniBand cluster transparently accesses the storage and Ethernet network. The highly scalable QLogic SilverStorm fabric ensures Cambridge has room to grow, while reducing costs and complexity in the network.
Exam Time
With the cluster installed and fine-tuned, it was now time to test the system using the LINPACK benchmark. The LINPACK benchmark is a standard used by the Top500 List in order to measure a cluster’s computing power. This is achieved by running a program to solve a dense system of linear equations. The resulting number of MFlops, GFlops, or TFlops (millions, billions, or trillions of floating point operations per second) represents a good measure of the system’s theoretical peak performance.
QLogic Makes the Grade
The LINPACK benchmark tests were run, and with 4 hours to spare, the Cambridge system running on a QLogic InfiniBand network yielded over 18 TFlops. This rate of performance was strong enough to earn Cambridge the # 20 spot in the November Top500 List of the world’s most powerful supercomputers. This ranking also puts the Cambridge cluster at # 7 in all of Europe and # 2 in the UK.
Just as important to making an A+ supercomputer was a commodity price tag. Cambridge turned to Dell and QLogic for a cost-effective high performance solution that makes the grade.
The InfiniPath InfiniBand host channel adapter yields the lowest latency, the highest message rate and highest effective bandwidth of any cluster interconnect available, enabling Cambridge to gain maximum advantage and return from their investment in clustered systems. In addition, SilverStorm InfiniBand switches and directors by QLogic ensure that the overall fabric maintains a low latency, high bandwidth, simple, yet highly scalable design with reduced operating costs, as well as up to 1/3 less power requirements.
“The InfiniBand network by QLogic, including HCAs and IB switches & directors provided us the lowest latency, high bandwidth, high performance interconnect we could achieve, all at an affordable cost. The ability for QLogic to implement the solution so quickly enabled us to reach a prestigious # 20 on the Top500 List.” Paul Calleja, Director of High Performance Computing,Cambridge University.
About High Performance Computing (HPC)
High Performance Computing (HPC) using clusters of inexpensive, high-performing processors is being applied to solve some of the most difficult computational problems in the world.
HPC techniques are used to tackle computationally challenging scientific work in universities and government agencies around the world, as well as in businesses where rigorous analytics are needed to model product characteristics and simulate various scenarios. QLogic is a leader in the HPC space with its portfolio of InfiniBand products, including host channel adapters (HCAs) and multi-protocol fabric directors and switches.
R S S feed
