Methodology in a Nutshell
Ken Biba, CTO of our testing partner Novarum, traveled to Atlanta, Boston, Chicago, Dallas, Denver, Las Vegas, Los Angeles, New Orleans, New York, San Francisco, San Jose, Seattle, and Washington, D.C. to test the performance of each of the four national wireless services (AT&T, Sprint, T-Mobile, and Verizon) at various locations of the cities. Our test cities were chosen for their varying population densities, physical topography, and cellular environments.
We tested the various wireless services at ten locations in each city—five indoors (often in a Starbucks) and five outdoors. We selected testing locations based on a grid covering the center of the city, but we did not test service in suburbs or in rural areas.
We asked the four wireless carriers to send us the 3G and 4G phones they thought would perform best on their data networks. For our 3G tests, we got a Motorola Atrix 2 (AT&T), an LG Marquee (Sprint), a Samsung Sidekick 4G (T-Mobile), and an HTC Droid Incredible (Verizon). For our 4G tests, we received an HTC Vivid (AT&T), a Samsung Galaxy S II (aka Epic 4G Touch, from Sprint), an HTC Amaze (T-Mobile), and a Motorola Droid Razr (Verizon).
On each phone, we used the FCC-approved Ookla testing app to measure speeds. At each testing location, we had Ookla send data to and from a server on one coast, and then to and from a server on the opposite coast; then we averaged the results. Especially with 4G service, we found that the farther the server is from the device, the higher the network's latency time (the time that passes before the transfer starts) is, and the slower the upload and download speeds are.
If no 4G signal was available, we recorded the speed of the fallback 3G network.
We chose our overall winners based on a weighted composite score that factored in the services' download speeds and their upload speeds. Each resulting speed number has a margin of error of plus or minus 5 percent.
Because wireless signal quality depends to a large extent on variables such as network load, distance from the nearest cell tower, weather, and time of day, we can't predict exact performance in a specific area based on our results. However, the results do illustrate the relative performance of wireless service in a given city on a given day.
Reading the Charts
The city-specific charts (like the one for Atlanta at right) report average 3G and 4G speeds--both for uploads and for downloads--for each wireless carrier in each of the 13 cities where we conducted testing. The bars on the left represent 3G speed data, and those on the right represent 4G speed data. The lower set of bars in each case shows the carriers' average download speed for the city, and the upper set reflects their average upload speeds.