Matt Corddry, Facebook’s director of hardware engineering, should be grateful to Tesla. Not because he drives one (he doesn’t), but because the popularity of its electric cars could help Facebook take a little more cost out of running its data centers.
Corddry runs Facebook’s hardware engineering lab, which designs the cutting-edge servers, storage gear and other equipment that power its services. It shares those designs with the outside world through the Facebook-led Open Compute Project, and one of the technologies on his mind these days is lithium-ion batteries.
Facebook has just started testing lithium-ion batteries as the backup power source for its server racks and plans to roll them out widely next year. Lithium-ion has been too expensive until now, Corddry says, but its use in electric cars has changed the economics. It’s now more cost effective than the bulky, lead-acid batteries widely used in data centers today.
“The crossover moment for us was really electric cars, especially when Tesla and the [Nissan] Leaf and those guys took off,” Corddry said this week, as he showed a reporter around the hardware lab at Facebook’s Menlo Park, California, headquarters.
“The inflection point has just happened in the industry where lithium-ion is cheaper to deploy than lead-acid for a data center UPS.”
The UPS, or uninterruptible power supply, is a critical piece of equipment in data centers. If there’s a power failure from the grid, the UPS kicks in immediately to keep the IT equipment running during the minutes it takes for a diesel generator to fire up and take over.
Lead-acid batteries—the type that start your car in the morning (assuming you don’t drive a Tesla, either)—are still cheaper per unit of energy than lithium-ion, but the recharging equipment and regular maintenance they require make them more costly overall, Corddry said.
They also take up a lot of space. Facebook designed its own lead-acid backup system, yet it still occupies a hulking cabinet 7 feet tall to provide backup power for just six racks of servers. Facebook uses thousands of those cabinets to run its services worldwide.
About a year ago, it started work on a high-density lithium-ion battery pack that slots directly into the server rack as if it were another piece of IT equipment, eliminating the cabinet completely. The battery cells it’s using are almost the exact same type you’d find in a high-end electric car, except the batteries Facebook uses are designed to discharge all their power in 90 seconds—enough time for the diesel generator to kick in.
“If Tesla built a dragster instead of a car that’s meant to go 300 miles, this is the battery they’d use for the Tesla dragster,” Corddry says.
The lithium-ion battery packs have other advantages, too. There’s one to each half-rack of servers, so if a battery pack fails the problem will be isolated to that group of servers. By contrast, in some large data centers, a single UPS supports hundreds of racks of equipment.
“Most applications are good at handling small scale-failures, but almost no apps are good at handling a massive concurrent failure,” Corddry says. “Big central UPSes can cause a massive concurrent failure. Once the UPS goes, you have no line of defense—you’re down.”
Putting the battery packs close to the servers is also more energy efficient, he says, because the power doesn’t traverse as many conversion points as with an external UPS. And he expects the lithium-ion batteries to have a slightly longer life span, though time will tell.
Facebook has just started pilot testing the battery packs in its data centers, he said. It runs a “synthetic load” for a couple of weeks to make sure the technology will hold up, then switches to live customer data after that.
“Our plan is to have our mass-production gear hitting the data center next year,” Corddry said.
He wouldn’t quantify the exact cost savings over lead-acid batteries, but if Facebook is making the change throughout its infrastructure, it’s safe to assume they could be significant.
If all goes well, there’s no reason lithium-ion won’t find its way into enterprise data centers. Facebook is designing the battery pack into the second version of the Open Compute Project’s power shelf, which includes all the power components for a server rack. The first version works with the big lead-acid battery cabinets.
Like other OCP projects, the goal is to come up with a more energy-efficient and flexible design than those offered by the major manufacturers. Any company can pick up the design and build it.
There are six battery packs in the v2 power shelf, including two redundant ones in case of failure, for a total 13.2 kW of backup power. That’s less than some high-performance computing centers need, but for Facebook—and most other businesses—it’s plenty.
“People think of lithium-ion as a premium, expensive thing, but this is actually cheaper for us,” Corddry said.