Google network engineer Irena Nikolova discussed the company-wide implementation at the Usenix Large Installation System Administration (LISA) conference, being held this week in Boston. There, she shared some lessons that other organizations might benefit from as they migrate their own networks to the next generation Internet Protocol.
From the experience, Google has learned that an IPv6 migration involves more than just updating the software and hardware. It also requires buy-in from management and staff, particularly administrators who already are juggling too many tasks. And, for early adopters, it requires a lot of work with vendors to get them to fix buggy and still-unfinished code. “We should not expect something to work just because it is declared supported,” the paper accompanying the presentation concluded.
“I think everyone who has tried to migrate to IPv6 has run into the same problems we have,” Nikolova said.
The project, which has been under way for about four years, turned out to be a larger endeavor than the engineering team had anticipated. It is only half way finished. But the company has made significant gains in this time. About 95 percent of Google’s engineers now have IPv6 access on their desks. Eventually, the company plans to have an IPv6-only network.
The project was started in 2008 by a small group of Google engineers, some of whom worked on it in the 20 percent of their work time that Google allots for its engineers to pursue their own pet projects. The goal was “IPv6 everywhere,” Nikolova said.
Part of the interest in the upgrade was practical. Even though it was a private network, Google’s internal network used public IP addresses, and Google was running out of internal IPv4 addresses. Also, Google engineers were developing IPv6 versions of Google’s own tools and applications and needed to test this software internally before releasing it to the public.
Google’s internal network spans more than 200 offices worldwide, serving about 30,000 employees. It consists of a wide variety of devices from companies such as Cisco Systems, Juniper Networks, and Aruba Networks, hundreds of commercial and home-built applications, and a range of operating systems, including Linux, Mac OS X, and Microsoft Windows.
The engineers modeled the IPv6 network as closely as possible on the existing IPv4 one, to keep the routing and traffic flow largely the same. At first they ran IPv6 over the IPv4 networks, a process called tunneling. Then, in cases where it was feasible, they set up dual stacks, where IPv4 and IPv6 run side by side. Eventually, they want to make the network IPv6 only.
To assign IPv6 numbers to devices, Google followed the guidelines in the Internet Engineering Task Force’s RFC 5375. Each campus or office got a /48 address block, which meant that it was allotted 2 to the 80th power addresses. In turn, each building got a /56 block of those addresses (or about 2 to the 72nd power addresses) and each VLAN (Virtual Local Area Network) received a /64 block, or about 2 to the 64th power addresses. To assign numbers to specific devices, the engineers used the Stateless Address Auto-Configuration capability (SLAAC), which allows the devices to assign numbers to themselves. This approach eliminated the need to manually assign numbers to each device. It was also necessary in that at least some operating systems do not yet support DHCPv6, the version of Dynamic Host Configuration Protocol server-based addressing mechanism for IPv6 networks.
One of the major issues was inadequate support for IPv6 in network devices and software, Nikolova said.
Many network devices now only support IPv6 in software, meaning that much of the traffic processing is carried out in software, rather than with customized hardware. As a result, IPv6 network operations consume more processor cycles than IPv4 operations do. At least one wireless equipment vendor did not support ACLs (access control lists). Also, the network’s WAN (wide area network) acceleration devices can not encrypt IPv6 traffic, because the protocol they use–WCCP (Web Cache Control Protocol)–does not yet work with IPv6. In addition to networking gear, printers remain problematic, in that many do not fully support IPv6.
Application and OS compatibility also proved to be a challenge. The company has been phasing out those applications that do not support IPv6, though many essential tools, such as databases and billing applications, remain in operation because they can not be modified or upgraded easily. And while the current versions of most OSes support IPv6, they do not do so by default, which causes additional work on the part of administrators and users.
“When it comes to technical problems, we can confirm that there is a lot of new, unproven and therefore buggy code, and getting our vendors aligned so that everything supports IPv6 has been a challenge,” the Google paper stated.
Google also faced challenges with service providers, those companies that provided network connectivity to Google’s offices. The SLAs (Service Level Agreements) were not as rigorous as those for their IPv4 services. They took a longer time connecting separate IPv6 points than they did with IPv4 peering sessions. Google itself had to rewrite its own network monitoring tools to work with IPv6 as well.
Despite these obstacles, Nikolova is confident that the team will achieve its goal of implementing an all-IPv6 network within Google’s offices.
“At some point, we stopped talking about IPv6 as the ‘new protocol’ and started calling IPv4 the ‘old protocol’,” Nikolova said.
Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab’s e-mail address is Joab_Jackson@idg.com