comScore Money-for-Real-Ranking Controversy Misses the Big Problem

When it comes to virtually anything on the web, people want to know how big an audience a site has. And there's been a big inside-baseball to-do in the tech industry over charges that comScore, used by many in business and the media as the way to measure traffic. The outrages and thrashing over what appears to be a pay-for-praise scheme, where companies must chip in if they wish to be measured with increased accuracy, is all well and good. But the discussion largely misses the issue that many in the media and even many in corporate management are constantly pointing to and making decisions based on all sorts of tech numbers that are largely nonsense.

For years, comScore has used a panel approach to estimating web traffic. It would work with panels of users, tracking what they do, and use that to extrapolate traffic data. The rating company had recently introduced a program by which either clients or sites willing to pay $10,000 could get a tracking bug inserted on their page so that comScore would have a record of actual traffic, and then use both that and the panel data

The current outbreak of criticism targeting comScore largely started with Jason Calacanis, who made a few claims:

• Many companies have been complaining for a long time that comScore was under-counting their traffic, because they could compare what the rating company claimed to their own traffic reports.
• comScore started a new approach: combining panel information with web measurements (presumably both to be sure that a site wasn't gaming the system. But it was only available to its regular clients, who paid a lot, or to those willing to spend $10,000 a year.
• Calacanis claimed that comScore offered him "a clandestine deal after I continued to publicly complain about their methods" to give him free use of the direct measurement system if he would stop criticizing the company.

But the fundamental issue for the industry and the media is not whether comScore was essentially trying to twist arms and browbeat executives into what was essentially rating payola. It is the question of why so many supposedly smart people keep swallowing utter nonsense as though it were revealed truth.

The comScore episode is just one example. But open any business magazine, or any business plan, and you'll see people tossing about completely unsupported figures. Why do so many assume that a relatively small and essentially self-selected panel could accurately represent the browsing habits of a nation? Why do so many look at U.S.-centric numbers, whether from comScore or Compete.com and assume that they give an accurate worldwide picture? For that matter, why do so many believe Apple's (AAPL) download numbers without questioning? Why do so many insist that you can discuss the quality of a digital camera by taking a megapixel count, although the quality of the firmware, other specifics of the image sensor, and even what a consumer intends to do with the camera can have a far bigger impact? Why does anyone act as though the claimed battery life of a laptop was anything other than a rationalized fiction? How could so many promote a wave of technology companies that became the bubble of 2001 when the thought of running a business without concern about income, let alone profit, was so completely absurd on its face? For heaven's sake, at this point, why believe that a company is actually going to ship a product on time when you can look at its success rate of accurate shipping predictions and give a fact-based probability? Why believe any tech company's pre-earnings guidance when history shows that it consistently low balls so it can then "beat" analyst expectations?

Too many companies have shown again and again that a huge number of executives with a financial state in the outcome will obfuscate and outright lie to manipulate people. Unfortunately, there are too many credulous people who recommend, promote, and supposedly vet technologies, companies, and investments on little more than waving a wetted finger in the air. Maybe it's time that a change starts with analysts and reporters taking a more critical look at the numbers they like to through into stories because it makes them sound as though there's some insight. Time for a reality check. Not only does the emperor have no clothes, but he never owned any in the first place.

Image via stock.xchng user Ayla87, site standard license.