Information Age: News, analysis & insight for IT & business leaders

Ask Jim Sterne about the challenges he faces as president of the newly formed Web Analytics Association (WAA) and, despite a decade since the birth of the World Wide Web, he says there is still a great deal of confusion about how customers' behave online and how that can provide invaluable information. "One of our biggest problems is that people still don't get web analytics."

The WAA was formed in February 2005 with the primary purpose of uniting web site owners with the vendors of web analytics tools. Those vendors' software is used to capture and analyse data about visits (and visitors) to web sites.

Even companies that successfully use web analytics tools to capture that data frequently find that their managers are not equipped to analyse it effectively. That, in turn, leaves them unable to answer vital business questions: who visits the site, how often do they visit, what do they buy and how might they best be tempted back?

Judging from the market data plenty of people do actually get it. Sales by web analytics tools vendors, such as WebTrends, Clickstream Technologies, Site Intelligence, CoreMetrics and WebSideStory, totalled $292 million in 2004, an annual rise of 31%. But the broader appreciation of web analytics must be addressed, says Sterne, if that buoyancy is to continue. In particular, organisations need to develop a better understanding of what the data generated by web sites means and what they should be doing with it.

The ease with which today's web analytics packages generate mountains of slick reports about traffic to web sites hides "an ugly truth", says Bob Chatham, of IT industry watcher Forrester Research: "The underlying numbers are tainted by numerous sources of error."

These sources of error include uncertainty over what constitutes a single visit and how individual users should be counted. These questions are likely to be further compounded by the increasing use of 'anti-spyware' - software that helps users avoid having their Internet use scrutinised by the blocking of software cookies. That means that a visitor with no cookie is assumed to be a new visitor, inflating unique visitor counts. For these reasons, users of web analytics tools are strongly advised by Chatham to make sure they fully understand the meaning of the statistics their applications generate.

That is no easy task, he adds. "Even leading packages such as Omniture and WebTrends can produce stats that make analysts question their sanity. Over the past few months, numerous Forrester clients have called to question why their new visitor counts seem to be growing at unexpected rates," he says.

The confusion is not helped by a lack of commonly used terminology among web analytics vendors, says Sterne, and developing standards in the terms and best practices used by the web analytics industry is one of key aims of the WAA.

The validity of web analytic tools, moreover, is further threatened by a corporate culture that treats reporting tools as records of the past, not guides for the future. Sterne believes that, in order to derive a return from their web analytics tools, businesses must learn to use them to ask new questions and test hypotheses rather than to periodically generate the same, standard answers.

"You don't buy a web analytics 'solution', you buy a web analytics tool," he says. "You have to use it actively, and learn how to take advantage of it to make business decisions."

The failure of many companies to spend more on training to instill this inquisitive attitude among end users has prevented them from achieving the return on investment for which they may have hoped. "To insulate their companies from costly mistakes made by over-zealous business users - and to protect the credibility of web usage tracking as a science - analytics heads must assume responsibility for educating their user bases on potential sources of error in web data," says Chatham of Forrester.

When used properly, says Sterne, the potential benefits of web analytics far outstrip the costs. "If you can optimise you web site to convert 2% of your visitors into customers, and you have a million visitors a day, that is a huge profit increase," he explains.

The more sophisticated functions of high-end web analytics tools can transform a web site from a shop front to a market research laboratory. Differing marketing messages can be presented alternately and the one that generates the most clicks identified, enabling the company to isolate the most successful message-based audience response.

The establishment of the WAA reflects the fact that there is an opportunity for web analytics vendors to establish their applications as indispensable tools for the modern enterprise, but that opportunity is dependent on their ability to manage users' expectations. To ensure that the current market buoyancy is sustainable, the web analytics industry must continue to educate the market place.