The Federal Communication Commission's Consumer Broadband Test is drawing criticism from skeptics who are calling it too simplistic and inaccurate to yield useful data.
The FCC this week launched the testing program on its Broadband.gov Web site to help consumers "test their broadband service and report areas where broadband is not available." Taking the test requires users to list the address they're accessing the Web from and also whether they are using the wWb at home or at a business. From there, the test measures the connection's download and upload speeds as well as its latency and jitter.
The test has drawn criticism, however, for being too simplistic and not taking several different variables into account. Lauren Weinstein, the co-founder of the People for Internet Responsibility advocacy organization, listed several factors on his blog that the FCC failed to account for, such as the fact that user testing data might become skewed if a user conducts a test during peak congestion hours. Additionally, Weinstein criticized the test for not providing users with information about its underlying server testing infrastructure.
"As anyone who uses speed tests is aware, the location of servers used for these tests will dramatically affect results," Weinstein wrote. "The ability of the server infrastructure can be quite limited depending on the ISPs' own network topologies."
Weinstein concluded that the tests employed by the FCC, which actually use the same testing system as the Speedtest.net Web site, were only useful for helping "categorize users into very broad classes of Internet service tiers" and not much else. This is best illustrated, Weinstein wrote, by the fact that when he conducted the FCC's test on his own connection, the test "showed consistent disparities of 50% to 85%."
Brett Glass, the owner and founder of the Wyoming-based ISP Lariat Networks, also criticized the FCC's speed test for being far too simplistic and said that it couldn't account for some of the smart routing techniques employed on his network to optimize user experience.
"My network routes different types of traffic through different connections, which are optimized for that type of traffic," he says. "But the test doesn't 'know' this. It tries to access random, uncacheable data through our cache and thus gives results which are not typical of user experience… In short, the tests are 'dumb' tests designed for 'dumb' networks."
The test is not without its defenders, however. John S. Quarterman, the CEO of Internet risk management firm InternetPerils, doesn't dispute the limitations of the FCC's current testing protocol, but rather says that the commission doesn't need fine-grained data in order to get a reasonably accurate picture of a typical user's broadband experience. The test will become valuable, he says, if the FCC gets a very large number of people to take it, thus allowing the commission to throw out statistical outliers such as tests taken during peak congestion hours or tests taken during network downtimes.
"Tests by multiple subscribers will give a pretty good idea of what that service is really providing," he says. "Even if some users test while somebody else is using the same connection, others will not, so you can get a pretty good sense of the maximum speed being provided."
Of course, the do-it-yourself consumer tests aren't the only way the FCC is trying to get a handle on broadband speeds in the United States. The commission Friday issued a notice that it is looking to contract with a third-party measurement company to "measure the speeds and performance of a representative, cost-effective, statistically relevant sample of U.S. fixed broadband households across geographies, technologies and providers." The FCC says this initiative will try out different approaches for
Both testing initiatives were launched over the past two days in the lead up to the release of the FCC's national broadband plan due to be released next week on March 16. The plan was commissioned by Congress last year to provide the government with a blueprint for building out broadband infrastructure in regions of the United States that lack access to quality, affordable broadband services.
Read more about lans and wans in Network World's LANs & WANs section.
This story, "FCC's Broadcast Test Met with Skepticism" was originally published by Network World.