The Scary Side of Virtualization
At the Computerworld Premier 100 IT Leaders conference in March, one CIO stood up to express his unease about the security of a virtual infrastructure that has subsumed more than half of his company's production servers. Two other IT executives chimed in with their own nagging worries.
None of the executives in that room wanted to admit on the record that they feel vulnerable, but Jai Chanani, senior director of technical services and architecture at Rent-A-Center Inc., feels their pain. "One of my biggest fears is the ability to steal [virtual servers]," he says.
How concerned is your organization with the issue of security in a virtualized environment?
* Very or extremely: 32.7%
* Somewhat: 36%
* Minimally: 23.7%
* Not at all: 7.6%
Source: TheInfoPro survey of 214 IT security professionals, November 2010
Chanani's team has about 200 virtual servers operating as file, print and, in some cases, application servers. But, for security reasons, his shop doesn't use virtualization for the company's ERP system, databases or e-mail.
Michael Israel, CIO at amusement park operator Six Flags Inc., voices a different concern. For him, the most unnerving scenario is a rogue administrator moving virtual servers from a secure network segment onto physical hosts in an unsecured segment, or creating new, undocumented, unlicensed and unpatched virtual servers. "The last thing I want is 25 servers out there that I don't know exist," he says.
John Kindervag, an analyst at Forrester Research Inc., says he's heard stories from clients who have had VMware's vCenter management console compromised, enabling the attacker to copy a virtual machine that can then be run to access data. "When you steal a VM, it's like you broke into the data center and stole a piece of hardware. It's potentially devastating," he says.
"We worked for many years with customers on best practices that make this a complete nonissue," says Venu Aravamudan, senior director of product marketing at VMware Inc. He says most users address such risks by following best practices such as creating an isolated network segment for managing the resources, and creating role-based access controls.
The migration onto virtual servers has saved businesses huge sums of money as a result of consolidation and improved efficiency, but as virtualization gobbles up more and more production servers, some IT executives are getting indigestion. Has anything been overlooked? Could a catastrophic breach bring down critical applications -- or perhaps an entire data center?
"Customers wake up one day, realize that 50% of their business-critical apps reside on virtual infrastructure and say, 'Gee, is that secure?' That's very common," says Kris Lovejoy, vice president of strategy at IBM Security Solutions, a security consultancy.
"There are some huge, well-known corporate names around the globe that you'd think would have this stuff pretty much beat. That couldn't be further from the truth," says Andrew Mulé, a senior security consultant in EMC Corp.'s RSA unit.
The problem isn't that a virtual infrastructure is difficult to secure per se, but that many companies still haven't adapted their best practices (if they have them) to the new environment.
Virtualization introduces technologies -- including a new software layer, the hypervisor -- that must be managed. Also new: virtual switching, which routes network traffic between virtual servers in ways that aren't always visible to tools designed to monitor traffic on the physical network.
Moreover, virtualization breaks down the traditional separation of duties within IT by allowing a single administrator to generate new virtual servers en masse at the push of a button, without approval from purchasing or input from the network, storage, business continuity or IT security groups (see "Beware the All-Powerful Admin," below).
Meanwhile, virtualization-aware security technologies and best practices are still evolving. The market has emerged so quickly that customers haven't been able to keep up from a best-practices standpoint, says Lovejoy. There's a lack of knowledge on the subject and a lack of skills in the field.
"The questions about security in a virtual environment are centered around lack of visibility, lack of control, and fear of the unknown," says Bill Trussell, managing director of security research at TheInfoPro, an IT market research firm in New York. Could someone hijack a hypervisor within a business's virtual infrastructure and use it to compromise all of the virtual servers residing on top of it -- as one CIO feared? Could an attacker breach one virtual server and use it as a platform to attack another virtual server, such as a payment-card processing application residing on the same hardware, without the administrator ever knowing about it?
Concerns about scary scenarios like those persist despite the fact that there have been no known attacks against virtual infrastructures, says Eric Baize, RSA's senior director for secure infrastructure.
When TheInfoPro surveyed 214 IT security professionals earlier this year, it found that one-third were "very or extremely" concerned about security in a virtualized environment.
Worries about an attack that could compromise a hypervisor rose after Joanna Rutkowska's "Blue Pill" hypervisor malware rootkit at a Black Hat conference in 2006.
Since then, however, the industry has moved forward with hardware technologies to ensure the integrity of hypervisors, such as Intel's Virtualization Technology for Directed I/O (known as VT-d). "Today, most of [Intel's] Core i5 and i7 processors have those technologies," and virtualization software providers have moved to support those features, says Rutkowska, founder and CEO of Invisible Things Lab, an IT security research firm.
Rutkowska herself doubts that anyone will actually use a Blue Pill-type rootkit to compromise virtual machines. "The bad guys don't really have any incentive to use such sophisticated rootkits," she says, especially since better-known rootkit technology from the '90s still works well for attacking traditional operating systems.
"People are wringing their hands over theoretical scenarios rather than ones that have been documented to be a problem," Trussell says.
Next page: Misguided strategies to avoid