Google Inc. has confirmed its use of data center containers at its first Data Center Efficiency Summit in Mountain View, California. Google provided an overview of how the containers were implemented in the company’s first data center project in the fall of 2005.
The servers are based on a proprietary design which has been kept under wraps for years. Most companies buy servers from the likes of Dell, Hewlett-Packard, IBM, or Sun Microsystems. But Google, which has hundreds of thousands of servers and considers running them part of its core expertise, designs and builds its own.
The servers are now in the sixth or seventh generation and feature a 12V battery on each server in lieu of a centralized uninterruptible power supply. This design ensures servers have continuous power, but results in lower costs and eliminates any wasted capacity. Efficiency is raised from 92-95% for a large UPS to over 99.9% for the individual battery design.
Each server is a 2U thick rackmount affair with dual processors and two hard drives. The motherboard is Google-designed and built by Gigabyte. Each board features eight DIMM slots and run either AMD or Intel x86 processors. Another surprise is in the power supply design which only supplies 12V power with conversions for 5 and 3.3V taking place on the motherboard. This allows Google to run the PSUs closer to rated capacity which means it runs at a higher efficiency. The design adds about $1 to $2 dollars to the cost of the motherboard but reduces the cost of the power supply.
The Google facility features a "container hanger" filled with 45 containers, with some housed on a second-story balcony. Each shipping container can hold up to 1,160 servers, and uses 250kW of power, giving the container a power density of more than 780W per square foot. Google’s design allows the containers to operate at a temperature of 81°C in the cold aisle. Those specs are seen in some advanced designs today, but were rare in 2005 when the facility was built.
Google’s design focused on "power above, water below,", and the racks are actually suspended from the ceiling of the container. The below-floor cooling is pumped into the hot aisle through a raised floor, passes through the racks and is returned via a plenum behind the racks. The cooling fans are variable speed and tightly managed, allowing the fans to run at the lowest speed required to cool the rack at that moment.
Google was awarded a patent on a portable data center in a shipping container in October 2008, confirming a 2005 report that the company was building prototypes of container-based data centers in a garage in Mountain View. Containers also featured prominently in Google’s patent filing for a floating data center that generates its own electricity using wave energy.
