World’s Most Expensive Computers

Posted by Sam Churchill on

The bi-annual top 500 supercomputer list was announced today at SC09 in Portland. A $104 million Cray XT5 at Oak Ridge National Laboratory beat out the previous number one spot from the $125M Los Alamos Roadrunner, while China entered the Top 10 with a hybrid Intel-AMD system, reports Computer World.

According to, the upgraded Cray XT-5 (Jaguar) at Oak Ridge, in Tennessee, is now top dog. The Cray XT5 now boasts a speed of 1.759 petaflops per second from its 224,162 cores, while the IBM Roadrunner system at the U.S. Department of Energy’s Los Alamos National Laboratory in New Mexico slowed slightly to 1.042 petaflops per second after it was repartitioned.

The Roadrunner at Los Alamos models nuclear explosions while Jaguar carries out scientific research on the globe’s climate.

IBM’s Roadrunner became the world’s first to achieve a petaflop, or one thousand trillion calculations per second on May 25, 2008. Roadrunner uses a hybrid processor architecture, based on both Opteron X64 processors from Advanced Micro Devices (AMD) and the IBM Cell Broadband Engine (Cell BE) processing elements.

The Cray XT5 at Oak Ridge increased the number of its Six-Core AMD Opteron processors to more than 224,000 and advancing its peak performance to more than two petaflops of compute power.

The list of the Top 500 supercomputers, released on Monday during the SC09 supercomputing conference in Portland, Oregon, is compiled twice a year and is now in its 34th installment. The total capacity of the systems on the new list is 27.6 petaflops, up from 22.6 petaflops on the previous list in June.

According to the Top 500 List, five computers have broken the petaflops barrier, three in the United States, one in Germany and one in China.

  1. Oak Ridge National Laboratory – Cray XT5-HE (Jaguar), Opteron Six Core 2.6 GHz, Cores: 224,162, Peak Gigaflops: 2,331, Power in KWatts: 6,950
  2. Los Alamos Laboratory – IBM Roadrunner – PowerXCell/Opteron, Cores: 122,400, Peak Gigaflops: 1,375, Power in kWatts: 2,345
  3. University of Tennessee – Cray XT5 – Opteron Six Core 2.6 GHz Cores: Cores: 98928 Peak Gigaflops: 1,028, Power in KWatts: N/A
  4. JUGENE – Blue Gene – Cores: 294,912, Peak Gigaflops: 1,002, Power in KWatts: 2,268
  5. National SuperComputer Center in Tianjin – TH-1 Cluster, Xeon E5540/E5450, ATI Radeon, Cores: 71680, Peak Gigaflops: 1,206, Power in KWatts: N/A

The No. 3 system, Kraken, at the National Institute for Computational Sciences at the University of Tennessee, could deliver a sustained 832 teraflops per second. This Cray XT5 supercomputer was ranked No. 6 in June, when it was rated at just 463 teraflops per second.

China’s fastest supercomputer ever, the Tianhe-1 in the city of Tianjin, achieved 563 teraflops per second for the No. 5 ranking. It uses Intel Xeon processors with Advanced Micro Devices GPUs (graphics processing units) as accelerators. Each node of the 71,680-core system has two Xeons attached to two AMD GPUs, according to the compilers of the Top 500 list. Tianhe-1 was built by the National University of Defense Technology for the National SuperComputer Center and is intended to provide high-performance computing services in northeastern China. Applications will include petroleum exploration and aircraft design.

Intel processors power 402 of the systems on the list, or 80.4 percent, up slightly from 399 in June. The IBM Power architecture is the second most commonly used, with 52 systems, down from 55. AMD’s Opteron family appears in 42 of the systems.

Most of the Top 500 supercomputers — 426 systems — now use quad-core processors. Only 59 use dual-core chips, and just four systems are based on single-core architectures. There were six systems on the latest list using IBM’s nine-core Cell Broadband Engine processor, also used in the PlayStation 3. Gigabit Ethernet is the internal interconnect technology in 259 installations, compared with 181 using InfiniBand.

Hewlett-Packard led in the number of systems on the list, with 210 supercomputers or 42 percent, compared with 185 for IBM. However, the IBM systems accounted for the most computing power, with 34.8 percent of total performance, down from 39.8 percent. HP held 22.8 percent.

As serious gamers know only too well, bragging rights are short lived. Before you can flip the switch something faster will be available — for half the price. Even worse, a $100 million computer does not include the cost of a 6-10 Megawatt substation, which might be considered optional at extra cost.

nVidia says their Fermi GPU, delivers the same performance of a traditional CPU-based cluster at one-tenth the cost and one-twentieth the power. Intel hopes their cell-like Larrabie will create a architecture for console games (and perhaps supercomputers) when in comes out next year. Larrabee looks like an x86 cluster node with 32 cores in its first implementation. Each core has a vector processing unit that augments the SSE instruction set. Convey does it with FPGAs using hardware-based, application-specific instructions to accelerate HPC applications.

Data centers are being overloaded with demand for power. Google uses The Dalles Dam, with its 1.8 gigawatt power output, to power their data center along the Columbia river. Google says a $1,500 server costs $8,000 when the power and air conditioning infrastructure is considered.

Every watt of power requires $1 to cool it. Power efficiency is a big thrust at SC09, here in Portland. The Datacenter of the Future booth at SC09 showcases energy efficient HPC Datacenters from diverse locations around the globe.

Posted by Sam Churchill on Monday, November 16th, 2009 at 12:58 pm .

Leave a Reply