Microsoft Still Bugged by Software Problems

The proof of the pudding is in the eating, goes the old adage: a piece of folk wisdom that Microsoft's chief executive officer, Steve Ballmer, hopes the company can apply to software development.

Ballmer, who worked for a cake mix distributor before joining Microsoft, wrote customers Tuesday to tell them how the company is using new error-reporting technology to improve the feedback it gets about the customer experience.

"This system is now built into Office, Windows, and most of our other major products, including our forthcoming Windows .Net Servers," Ballmer wrote. "It enables customers to send us an error report, if they choose, whenever anything goes wrong."

By testing the company's recipes as they are served up to the customer, rather than as they are cooked up in the software lab, he hopes users won't crunch on a bug and be left with a bitter taste in their mouth.

Reporting the Problem

"Freeze-ups and crashes can be incredibly irritating, but rarely do customers contact technical support about them; instead, they close the program. Even when customers do call support and we resolve a problem, we often do not glean enough detail to trace its cause or prevent it from recurring," Ballmer said in his e-mail.

Since deploying the new error-reporting technology in its Office XP software suite, the company has made some interesting discoveries about how bugs affect users.

"One really exciting thing we learned is how, among all the software bugs involved in reports, a relatively small proportion causes most of the errors. About 20 percent of the bugs cause 80 percent of all errors, and--this is stunning to me--one percent of bugs cause half of all errors," Ballmer said in his e-mail.

Using this information about which bugs bite users the most, "We're now able to prioritize debugging work on our products to achieve the biggest improvement in customers' experience," he said.

The approach is paying off, he said: "Error reporting helped us to eliminate more than half of all Office XP errors with Office XP Service Pack 2."

Privacy Protection

Not everyone will appreciate their computer sending Microsoft details of what it was doing when it crashed, a fact of which Ballmer seems well aware.

"There are risks in offering this option to have software 'phone home' like E.T. One risk is that error reporting could compound a customer's irritation over the error itself," he said.

To limit this irritation, Microsoft developed a compact report format it calls the "minidump." A URL in Ballmer's letter links to a Web page explaining that the minidump contains information about the instruction the computer was executing when it crashed, the processor model, and operating system version running, a list of the software modules loaded, and details of all the program threads running at the time the error occurred, including the contents of the software stack.

A greater worry for many users is that Microsoft will use the information returned to spy on them.

"Customers may wonder what we do with their reports and whether their privacy is protected," Ballmer said, going on to explain, "We use advanced security technologies to help protect these error reports, which are gathered on a cluster of dedicated Microsoft servers and are used for no other purpose than to find and fix bugs."

A Quick Fix

With so many different versions of its operating systems out in the field, some of them patched, some not, Microsoft might already have found a fix for the errors reported. If so, users will be directed to a Web site "where they can learn more about and even fix the errors they report," Ballmer said.

This generation of error-reporting tools allows Microsoft engineers to listen to the complaints of users' machines, but the company is developing feedback tools to better listen to humans too.

"We're trying to create easy ways for customers to send us more nuanced feedback about their experience with our products--not only about crashes, but also about features that don't work the way or as easily as people would like," Ballmer said.

The company is making changes to its support service in order to solve customer problems more quickly, and will soon announce a new policy designed to make clearer its support for its products throughout their life cycles, he said.

Ballmer's e-mail message to customers is the second such from a senior Microsoft official. Bill Gates wrote the first, back in July, on the theme of trustworthy computing.