With the overview of our four contestants out of the way, let’s get down to business. To find out which browser is worthy of your bandwidth in 2020 we used a variety of testing tools.
We largely stuck to our typical testing regimen, but that is set to change in upcoming browser showdowns. A number of the tests we use have been unsupported for years, and it’ll soon be time to shake up our testing approach.
In addition to JetStream 2, we used JetStream 1.1 to maintain comparisons to previous runs of the showdown one last time. We also kept the now-unsupported Octane 2.0 and SunSpider 1.0.2 benchmarking tools. Then we turned to WebXPRT 3 and Speedometer to test the browsers under simulated web app workloads.
Finally, we took a look at CPU and RAM usage by loading a set of 20 websites in a single window in quick succession. Once all tabs began loading, we waited 45 seconds, and then checked the CPU and RAM usage. The idea was to see the amount of system resources the browser used during a heavy workload.
For the tests we left each browser in its default state. There are no extensions running, no account sign-ups, nor was there any deliberate tinkering with settings. It’s just raw browser action.
Our test rig was an Acer Aspire E 15-575-33BM laptop loaded with Windows 10 Home, version 1909. The laptop also has a 1TB hard drive, 4GB RAM, and an Intel Core i3-7100U. Each browser was tested over an ethernet connection.
The performance picture
Firefox came out of this showdown a browser of extremes. As we’ll see, it either wins a particular performance test or comes in dead last. This is most likely a result of our current browser monoculture. When a Chromium browser wins a particular category, its two cousins aren’t far behind. Nearly every Chromium-based victory was one of degrees between the three amigos, which inevitably pushes Firefox towards the back.
For all of the benchmarks, we ran the test three times in succession, except for WebXPRT 3 and our homegrown 20-tabs torture test. Each time we ran the test we relaunched the browser window. We then took the average score from the three runs.
Let’s start with JetStream 2, the newest addition to our testing regimen. Our top winner for this round was Edge, but as we said it’s hardly a resounding victory with Chrome and Opera not that far behind. Firefox performed abysmally in this test compared to its counterparts.
Looking at both JetStream 1.1 and SunSpider, Firefox won top marks. It won the SunSpider test handily (lower is better) with Edge coming in second. Interestingly, Edge was quite a few seconds faster than the other two Chromium browsers. Firefox was marginally better than in our 2019 outing, shaving 8 milliseconds off its score—though that is within the margin of error. Chrome and Opera weren’t even close to Firefox, but they did cut down their scores by more than 100ms. So good job to the V8 team.
Firefox’s JetStream 1.1 score (higher is better) wasn’t any better this time around than its previous runs, but the other browsers either didn’t improve enough (Chrome and Opera) or changed browser engines (Edge). That puts Firefox at the top spot for not sucking as much as Chromium.
For Octane 2.0, which is also no longer supported, Edge grabbed another victory. Notice, however, that it’s only marginally better than Chrome. Opera, meanwhile, was out to lunch compared to its brethren, and Firefox was pushed towards the back. Mozilla’s score is about the same as it’s always been, but the Chromium browsers improved significantly compared to the 2019 outing.
Moving on to the more modern Speedometer test, which quickly iterates through a set of HTML 5-based to-do lists. Chrome came out on top, with Edge a close second and Opera nearly three points behind the leader. Firefox, meanwhile, was way behind team Chromium. It’s worth noting, however, that Speedometer scores for Chromium browsers were much higher just two years ago with 90+ scores being common.
The numbers were much closer for WebXPRT 3, which looks very similar to WebXPRT 2015. The scores, however, are dramatically different. WebXPRT 2015 scores were commonly three hundred or more. With WebXPRT 3 the scores didn’t crack 150. That means comparing these scores to previous runs won’t give us any insights.
WebXPRT uses a wide number of web apps, from photo collections to online note-taking to data sets to score performance. This test is kind of like a PCMark for browsers, and to our mind, one of the most significant tests. Firefox came out on top here by a solid margin, with Edge coming in second followed closely by Chrome and Opera—the latter two tied for third. Again we’re seeing the Chromium effect on these scores.
Finally, we come to the memory and CPU tests. Slamming an average PC with 20 tabs of mostly media-rich sites all at once is going to chew up a good chunk of CPU and memory. Most of these browsers did not disappoint in that respect.
Edge was the best performer in CPU usage by a good margin. Opera came in second, followed hot on its heels by Chrome. Mozilla’s performance was a little better than last time, but still nowhere near what it needs to be.
This test is where each browser distinguished itself with finally at least three different outcomes.
Edge took the crown for memory as well, but Chrome wasn’t that far behind. Interestingly Opera was only marginally better than Firefox. Again, mostly different outcomes for each browser. The Chromium underpinnings matter less here.
And the winner is...
So who wins? Here’s how we see it.
Chromium Edge wins our top spot for a good showing in the stress test, as well as Octane 2.0 and JetStream 2. Chrome is a close second since its showing was consistently good and not far off of Edge. Really, this could’ve been called a tie as well, but the margins in some of the tests, while not wide, were enough to hand the crown to Edge.
We’re giving Firefox third place for strong showings in the old JetStream 1.1 test, as well as Sunspider and WebXPRT. We also feel compelled to be perhaps a little unfair and send some ire Mozilla’s way.
As the only non-Chromium mainstream browser it simply has to do better—especially in CPU and memory management. The new Quantum versions of Firefox are dramatically better than their predecessors, and as we said last time, if the stress test had gone better it might have taken the top spot or at least second place. PC users need a strong alternative choice to this Chromium soup we’re swimming in, and for that reason Firefox must up its game. To borrow a quote from Princess Leia, “Save us Mozilla, you’re our only hope.”
Opera comes in last for scoring either third or fourth place in every test we ran, save one. Granted, those losses were almost all a matter of degree, but it shows a consistent pattern.
To sum up: Edge is the best browser available right now, though Chrome’s performance is very close to it. Firefox is still a solid option if you want something that isn’t built with Chrome DNA. Finally, if you love Chrome but want something with a little more novelty then Opera is for you.