A sequence of errors led to Dell’s delivery of motherboards with malware and the company is in the process of overhauling its testing process to resolve issues before dispatching hardware to customers, it said on Thursday.
Dell on Wednesday said that some replacement motherboards for PowerEdge servers may have contained the W32.Spybot worm in flash storage. The malware issue affected a limited number of replacement motherboards in four servers, the PowerEdge R310, PowerEdge R410, PowerEdge R510 and PowerEdge T410 models, the company said.
“There was a sequence of human errors that led to the issue, That being said, we have identified and implemented 16 additional process steps to make sure this doesn’t happen again,” said Dell spokesman Jim Hahn.
Hahn did not provide additional details on the steps being added to track and resolve such issues. But he said that all affected motherboards had been removed from the service supply chain. Current antivirus software with updated signatures would flag the malware’s presence and users would have to be running an unpatched version of Windows 2008 or an earlier version of the OS.
A Dell quality management specialist wrote in an e-mail that the code was accidentally introduced during the manufacturing process of the server motherboards. The code was detected on the embedded server management firmware during internal testing by Dell.
Motherboards come with flash storage — typically NOR flash — that stores the BIOS, which provides the instructions to boot the system, said Gregory Wong, president of Forward Insights. Flash is a nonvolatile form of memory that can retain data even after a computer is shut down.
Flash on motherboards are susceptible to the same kind of malware infections that USB flash devices are prone to, said Simha Sethumadhavan, assistant professor of computer science at Columbia University. This incident shows how hardware, either flash or a processor, if hacked, can be used as a way to transmit malware.
“All software runs on hardware. If the processor is hacked then it can subvert all software countermeasures. Since hardware is the root of trust, attacks on hardware are potentially more dangerous,” Sethumadhavan said.
Motherboards also could have solid-state drive units for data storage, said Jim Handy, director at Objective Analysis, a semiconductor research company. But instructions to start a system originate from a NOR flash chip, which would also imply that the malware is “pretty small,” Handy said.
“This flash is the one that holds your BIOS and it can be updated online. If proper security precautions are not in place, the flash chip is every bit as capable of containing a piece of malware as is the hard-disk drive,” Handy said.
Columbia’s Computer Science Department uses Dell PowerEdge R410, but was not affected by the issue, said Daisy Nguyen, IT director for the computer science department at Columbia University. Nguyen said Dell offers competent products and the malware issue won’t affect the department’s decision to purchase products from the company.
“Dell immediately admitted to the problem,” Nguyen said. The company also moved quickly to resolve the issue, she said.
Nguyen said that Dell has agreed to try and send some samples of the motherboards with malware so the university can research the issue as part of an investigation into securing hardware systems.