Scott Culp is program manager for Microsoft's
Security
Response Center, which investigates reports of security holes
in Microsoft products and oversees the creation and distribution of patches and
security bulletins for users. We spoke with Culp about the increasing problem
of security holes in software and about the steps Microsoft takes to test
products before releasing them.
PCW: Tell me what Microsoft does to produce secure
software.
Culp: You start off with security in the design. Then
you're relying on good coding practices and on compiling tools to help you
catch as many errors as you can. Once implementation is done, you have testing
of the whole.
PCW: Is there a built-in timeframe for testing?
Culp: Actually there isn't. That became implemented with
Windows 2000. What we did with Windows 2000 was we said, if it says "security"
anywhere on the bug, then ship-schedule be damned.
PCW: Are you saying that in the past, you shipped
previous versions of Windows with known security bugs?
Culp: I wouldn't say that, but what I would say is that
there are always trade-offs between the schedule and the quality bar. Any
software vendor is going to make a call at some point about: Have we achieved a
sufficient level of quality? And understand that you'll never get to zero bugs.
At some point you have to cut the cord and ship. What we said with Windows 2000
was that we were exempting security bugs from that engineering trade-off. The
right number is zero, and we won't ship until we get down to zero.
PCW: But from that statement about Windows 2000, one
would assume that in the past you
did ship with known security flaws.
Culp: There are different degrees of security bugs. [You
have to ask] is this a bug, for instance, that allows someone to take control
of a system? That's obviously a serious bug. Is this a vulnerability that can
only be exploited if I can put my hands on the machine that I want to attack or
can I attack it from across the Internet?
In the past what we did was put all of those factors together and we
said it is acceptable to ship with a certain number of low-severity,
very-difficult-to-exploit security vulnerabilities. Starting with Windows 2000,
if it says security on the bug, it's got to be fixed before it goes out.
PCW: If you're now taking more time to test, why can't
your staff of well-trained testers and coders catch the bugs that hackers and
bug hunters catch?
Culp: Accurate security testing is very difficult--not
just for Microsoft but for anybody. You can't rely on code review alone to know
that you've got the security of a system right. The only thing you can do is to
try to emulate usage patterns and identify bugs that way. There are people who
will use the product in ways that we just didn't conceive. That's how many of
those bugs are found, in very unusual scenarios.
PCW: But hackers tend to try the same types of exploits
over and over again. Why can't Microsoft emulate the way hackers think and
figure out what they would do with the product?
Culp: We actually do.
PCW: Then why aren't you finding the bugs that they're
finding?
Culp: There are always going to be tests that we miss,
there are always going to be implementation errors that we're making. We learn
from people like [hacker and security consultant] Rain Forest Puppy; we work with them to find out what they're trying that we
haven't been trying. We incorporate that into what we're doing moving forward.
What we do that is different from virtually every other vendor is that
we don't hide them. If we make a mistake, we tell the whole world about it.
PCW: Isn't it more the case that you've been forced not
to hide it? Without full disclosure about holes, Microsoft in fact, has been
silent in the past about problems with its products.
Culp: Not for an awful lot of years. The response center
is about four years old. You can make an argument about what our practices were
before then, but the whole point in creating the response center was
recognizing our responsibility to our customers and that our position in the
market incurred a certain amount of extra responsibility.
I can give you a couple of examples. We had a vulnerability--I think it
was in
bulletin
#36--that could allow someone to change a password under some
fairly restricted conditions on a Windows 2000 server. The way we learned of
that issue was that a customer sent a note to Russ Cooper, who runs the NT
BugTraq
mailing list, and said he found something that looks like a security
vulnerability. Both Russ and the customer said, "I don't think you should make
it public. You should fix this thing transparently in the next service pack and
include this fix as a patch for a different issue." But if we fold this issue
into a patch for a weenie issue, there are people who are going to read the
bulletin and say "I don't need it" and they're not going to get the protection
that they need.
The second example is one that we found internally, in Windows 2000. We
were the only ones who knew about it. We fixed it in Service Pack 1 for Windows
2000, and a lot of people were picking it up. But there are people who don't
pick up the service packs. For instance, they may have long testing
requirements in their organization before they roll a service pack out into
their network. And we decided that this vulnerability was sufficiently serious
that we would rather write a bulletin, issue the patch, and tell the world
about a vulnerability that no one in the world knew about, rather than taking
the chance on some customers not getting the service pack.
PCW: There are some bug hunters who say that when
they've reported vulnerabilities to Microsoft, the company has ignored
them.
Culp: We answer every e-mail we receive at
secure@microsoft.com. ... Sometimes we check out reports and we say I'm sorry,
this is not a security vulnerability. And sometimes they disagree with us. And
then we have a discussion. Sometimes we wind up agreeing to disagree. But we
never ignore any report that comes in to us.
PCW: There is concern among the public that there are a
lot of safety regulations for other products on the market--drugs, cars, and so
on--but that there aren't the same kinds of quality and safety standards
applied to software.
Culp: Even something that we've been doing for a very
long time still has a failure rate. Even cars, that we've been building for
well over a hundred years, still have problems every so often with the
engineering. We've been building operating systems and computer software
packages for about 35 to 40 years. Not very long at all. And they are the most
complex things that humanity has ever tried to build.
PCW: A rocket scientist might disagree with that.
Culp: One of my computer science professors was really
fond of a study that announced the three most complicated things that humanity
had done at that time were: Number three was the Apollo moon shot, number two
was the Bell switching system, and number one was the PL1 compiler.
[Programming Language #1; a compiler is a program that translates a high-level
computer language into instructions a PC executes.]
PCW: So if operating systems are the most complex things
that humanity has ever built and we can't expect them to be secure, then what
do we do?
Culp: You build a response process like what we've got.
You say I'm going to make this product as good as I can. I'm going to
constantly improve my engineering practices and be state-of-the-art in terms of
engineering and in testing my product. And I'm going to recognize that I'm
never ever done--that this product, during its entire lifetime is going to have
additional bugs, and I've got to repair them.
In Office XP there are two features that are also going to be in Windows
XP. If Office crashes on you, it says it would help Microsoft build better
products if you wouldn't mind sending data on this crash to Microsoft. It won't
send any personal information. That will let us figure out what the problem is
and make the product better.
The second thing we do is that we have an automatic update that comes
out every month and includes all of the fixes that we've found because of all
of those bug reports that are now being automatically sent in. So once a month
you can get an automated notice that says we've got the July Office XP update,
would you like it? If you want to see everything that it fixes, here it is;
otherwise, if you just want to take it, just say yes and we'll drop it down.
We're heading toward the idea of self-healing software.
