For the fourth year in a row, we hit the road and tested mobile wireless network speeds across the country, putting the four largest mobile networks—AT&T, Verizon, Sprint, and T-Mobile—head-to-head. Our approach to testing wireless service has always been to closely replicate how people use wireless service in the real world, and then measure how well it really works. That’s why we do the testing ourselves, instead of crowdsourcing the testing data, as other studies often do exclusively.
We take great care to measure the various services under similar and repeatable conditions: We use the same devices, so that the comparisons we make afterward really say something about the relative values of each.
Our wireless testing methods and madness
We chose our test cities for their varying population densities, physical topography, and cellular environments.
Our 20 testing cities were Ann Arbor (Michigan), Atlanta, Boston, Chicago, Dallas, Denver, Houston, Kansas City (Missouri), Las Vegas, Los Angeles, Miami, New York, Omaha, Phoenix, Philadelphia, San Diego, San Francisco, San Jose (California), Seattle, and Washington, D.C.
We tested the wireless services at ten locations in each city—five indoors (often in a Starbucks or a similar place) and five outdoors. We selected the testing locations based on a grid covering the center of the city; we did not test service in suburban or rural areas.
At each testing location we tested the download speed, upload speed, and network latency of the 3G and 4G/LTE services of AT&T, Sprint, T-Mobile, and Verizon.
We tested using an automated tool developed by OpenSignal, which sends files to and from an Internet server to measure speed and latency. The servers are located in the same region as the testing city, so as to minimize variance caused by distance to the server. We tested each location until we got eight valid tests for each metric—download speed, upload speed, and latency—for each service.
We carried out the testing on weekdays only, between the hours of 8 a.m. and 6 p.m.
If at a given testing location we could not receive 4G/LTE service after several attempts, we did not use a fallback 3G-service speed or enter a score of zero when tabulating the carrier’s average speeds. Instead, we made sure that the connection failure was reflected in a separate “Reliability” average score.
To measure network latency, we pinged google.com servers three times for each service at each testing location.
We chose our overall winners based on a weighted composite score that factored in the services' download speeds and upload speeds. Each resulting speed number has a margin of error of plus or minus 5 percent.
Because wireless signal quality depends to a large extent on variables such as network load, distance from the nearest cell tower, weather, and time of day, we can’t predict exact performance in a specific area based on our results. However, the results do illustrate the relative performance of wireless service in a given city on a given day.
This story, "How we tested the nation's networks" was originally published by TechHive.