Let's Declare a Moratorium on Browser Speed Claims

Apple is touting its new Safari 5, is "the world's fastest browser." Over at Computerworld, Richi Jennnings has rounded up a bunch of blog posts that politely disagree. (Actually, one isn't so polite: it calls Apple's claim a "flat-out lie."

Who's right? Everybody and nobody, and that's the problem. I don't think Apple cooked its numbers, but the only ones it's published are for tests performed on a Mac, so they won't tell you anything about how Windows browsers compare. And even if you only care about Macs, the race is tight enough that different hardware setups will yield different winners.

The fact that Apple's published tests are for the Mac means that it didn't test Internet Explorer. If it had shown Windows results, Safari 5 would have beaten IE 8 to a pulp. Microsoft's Brandon LeBlanc posted a video showing IE trouncing Safari-but it's IE 9 that's doing the trouncing, and the "platform preview" of IE 9 is such an early draft that it isn't really a browser yet. (It's a promising rendering engine without a front end.) LeBlanc isn't arguing that Apple erred in not including IE 9-and is probably just as glad that it didn't use IE 8-but it just goes to show that there's no canonical set of browsers that deserve to be referenced in these tests.

Your conclusions about browser speed will also vary depending on whether you contrast Safari 5 with Google's Chrome 5 (the official current version) or Chrome 6 (the widely-available beta). And different benchmark suites show different browsers in the lead-which confirms that no benchmark is a definitive measure of how the browsers compare.

Apple's own tests show Safari and Chrome neck-and-neck: It has Safari beating Chrome in the influential SunSpider JavaScript benchmark by all of seven milliseconds. I don't take that result as proof that I should run Safari 5: I take it as evidence that it's a wash. Both Safari and Chrome are really fast browsers.

Browser companies obsessively trying to win benchmark tests is a great thing if it leads every developer to work really hard to make a fast browser-which, judging from both Apple's tests and ones with conflicting results, it has. But browser users shouldn't take these benchmarks very seriously. The one test that matters is a straightforward one that no browser company can perform: Does your browser feel fast to you? And with every browser only a free download away, there's absolutely no reason why you can't run this simple test on all of ‘em.

Subscribe to the The Advisor Newsletter

Comments