The Dirty Truths About Web Traffic

One of the most difficult things when you try to evaluate a website is the lack of reliable metrics. A related problem is determining which metrics really matter.

Into this void, many a privately held company exaggerates its traffic, and -- depending which metrics service they choose in their SEC filings -- public media companies may be doing so as well.

It is hard, as one who regularly examines these filings, to understand why the ComScore traffic reports used by most major media companies, including The New York Times, seem to regularly report higher traffic than do Compete, Alexa, Google Analytics, and some of the other services.

One of the best and clearest analysis of this audience measurement issue was published by Jason Pontin in Technology Review earlier this year. Here are a few highlights from his long, detailed piece:

• "No one really knows how many people visit websites. No established third-party supplier of audience measurement data is trusted. Internal web logs exaggerate audiences."
• "The issues involved are technical, and occluded by ugly jargon, but they concern anyone anxious about the future of media as print and broadcast television and radio shrink in importance."
• Internal company logs typically over count users who use more than one computer or browser, or those who empty their "cookies" frequently. They may also be counting all of the "bots" that crawl the web as if it were one big spider web.
• The large brand media websites are less affected by this problem than smaller operations, because when it comes to display ads, media buyers purchase packages of ad impressions and pay on that basis, as opposed to evaluating actual audience size.
• "(T)he real consequence of the audience measurement problem is a chilling effect on the transfer of advertising from older media to new."
• Meanwhile, search (keyword) advertising is growing rapidly (exhibit A: Google), and now accounts for 45 percent of all online advertising. "(T)he effectiveness of keywords is unambiguous: Advertisers pay directly for click-throughs or purchases. There's no need to appeal to anything so disputed as the size or composition of web audiences."
• Search advertising revenue grew five times as fast as display ad revenue in 2008, which is why publishers are losing the race for ad dollars online to Google, Yahoo and Microsoft.
• Finally, there's a structural problem with online advertising: "(T)he absence of commonly accepted, automated means to create, sell, serve, and track the performance of online ads."

Pretty much every day of the week, I speak with media industry execs who offer me traffic figures for their sites. If they are startups, it figures that they are probably citing their internal logs, which have the inflationary tendencies cited in Pontin's piece.

I can try to triangulate, using Compete, say, and checking what the execs tell me. Sometimes, a CEO will cite a traffic ranking that, if accurate, would make that site larger than the nytimes.com. When I check, however I can't find any evidence that the company in question company has even half that much traffic.

At that point, my options are to (1)take the CEO at his word; (2)publish that figure and contrast it with the measurements that the independent services provided; (3)ignore the traffic issue altogether.

After all, when it comes to talking about web traffic, the title of Pontin's piece got it precisely right: "But Who's Counting?"