NCTA Criticizes comScore Broadband Speed Test Data

ComScore broadband test results understate the speeds offered by ISPs, the NCTA said in a letter to the chief of the FCC’s Consumer and Governmental Affairs Bureau. The association said it sent the letter in response to assumptions about retail broadband speed disclosures in the National Broadband Plan that relied on comScore figures. “The comScore data cited in the plan suffers from a variety of problems and should not serve as the basis for concluding that there is such a significant gap between maximum speeds and ‘actual’ speeds,” NCTA Vice President Neal Goldberg wrote Joel Gurin, the chief of the Consumer and Governmental Affairs Bureau.

Concluding in the broadband plan that “actual performance experienced by consumers often is much less than the advertised peak speed,” the commission cited only a comScore database “on file with the FCC” that wasn’t publicly available or peer-reviewed, Goldberg wrote. An NCTA member bought the data at considerable expense from comScore and the trade association commissioned NetForecast to review it, the letter said. NetForecast identified five problems with the way comScore collected the speed test numbers and one with the way it interpreted it, it said.

The problems in data collection include rounding off the size of a megabyte, only using one TCP connection during speed tests and not taking account of constrained resources on users’ computers, the report said. “All the data gathering errors result in an underreporting of the actual speed delivered by an ISP on its network, and the individual errors create a compounding effect when aggregated in an individual subscriber’s speed measurement,” the study said. “The result is that the actual speed delivered by each ISP tested is higher than the comScore reported speed for each result of every test.” A problem with the way that comScore identifies which ISP tier its testers subscribe to further overstates the disparity between the speeds advertised and delivered, the report found.

The NCTA said it supports the plan’s recommendation that the commission contract out a report on broadband performance. Responses to the FCC’s request for quotations for the contract were due last week. But before the FCC settles on the testing procedure, it should get more input on the process from ISPs, the association said.

The NCTA also endorsed a recommendation in the plan that the commission work toward developing industry standards for measuring broadband performance and ways to do it. It urged the FCC to quickly set up a Broadband Measurement Advisory Council (BMAC) made up of industry and consumer representatives. “The Commission should not rule out the possibility that voluntary compliance with standards established by the BMAC would eliminate the need for mandatory federal standards,” Goldberg wrote in the letter.