“We’re not worried, it validates our strategy.” This is how almost all IT suppliers greet the entry of a big predatory gorilla company into their market. And almost every time, the stock price plummets (if they have one), and analysts begin to write their obituaries. Mostly, they are right.

So when Google, the search engine giant, announced it was entering the market for online analytics software back in late 2005, it seemed like the old ritual was being played out again. And when it later said its new product would be given away free, the smiles on the faces of the CEOs and marketing people at the rivals began to look like the first stages of rigor mortis. Commoditisation and consolidation seemed inevitable.

The only thing: it didn’t work out that way. In 2006, the market for web analytics has exploded. All the leading suppliers are reporting strong growth, with enterprise customers ignoring Google’s free offer and committing to projects that will cost $200,000 to $500,000 over the next five years. A small army of new consultants in web analytics is being trained up.

And, as David Pool of the web analytics company Site Intelligence points out, this is customer driven, with the world’s leading online and offline retailers leading the way. “We’re pitching to people with job titles that didn’t exist two years ago.” Sitting alongside the CIOs and IT specialists now are online affiliate managers, online business performance managers, customer insight managers, digital marketing managers, and search engine marketing specialists. At many big retailers, online analytics is falling under the remit of a “multi-channel marketing director” – an executive tasked with bringing online and offline sales and marketing campaigns together.

In some cases, whole teams have been created. Tesco, for example, not only has a “digital insight team” that spends most of its time analysing consumer behaviour on its Tesco.com websites, but has also created a joint venture, Dunhumby, that analyses customer and store card owner’s behaviour. “Everything they [Tesco] do is driven by data,” says Pool, whose company supplies analytics tools to the retailer.

At leading online travel site Lastminute.com, it is the same: “We have six core analysts working all day on site analytics,” says Duncan Horton, marketing director. But Lastminute’s use of its analytics package, from US hosted service provider Omniture, extends far beyond a specialist analyst team: “Our licences run into the hundreds in the UK alone. Omniture is not only our main analytics tool, but also our daily management information system”. As he looks out over the open plan office, he says, he can sees the analytics service on most of the screens he can see.

DIGITAL EXCELLENCE

What is driving all this? Google’s influence, say consultants, owes little to recent “validation” of the market, but is considerable nevertheless. As the power of good search engine rankings became apparent from 2001 onwards, search engine marketing (SEM) businesses sprang up to help businesses build traffic. Some of their methods were questioned, some were underhand, and some irked Google, but companies that had invested heavily in their web activity learned a valuable lesson: there is a science to online marketing, just as there is in, for example, positioning and then laying out a supermarket in order to maximise sales.

They didn’t have to look far to find the tools. Business analytics companies such as SAS Institute and Information Builders, were already advocating the use of their warehousing tools as a platform for analysing both on and offline data. They were joined by specialists such as UK-based Site Intelligence, which uses algorithmic techniques originally developed for molecular modelling to analyse visitor behaviour on big websites; and by dozens of suppliers offering web visitor reporting tools, from the market leader Webtrends, and WebsideStory, through to little known but rapidly growing companies such as Omniture. (See boxes on how web analytics works, and the web analytics market).

Many of these tools have been available for five years or even longer, but only in the past two has the market entered the so-called “tornado” phase. “In the last two years, there has been a revolution in the way businesses approach their web activity. The executives are saying: “This website is costing us a fortune. We need to know what we’re getting out of it,” says Conrad Bennett, European technical services director for WebTrends, the global leader in online analytics software by revenue.

Furthermore, he says, no one is content with analysis alone. “All of this is a complete waste of time unless we can take some action,” says Bennett.

That rush to action has been further inspired by some very high profile successes at some of the largest ecommerce and web operators – the so-called “Gods of ecommerce”. These companies – Amazon, eBay, Yahoo, and in the UK, Tesco, the BBC, Argos, and Lastminute.com – use web analytics extensively.

One of these is Amazon.com, which, says Jonathan Opperman, head of eBusiness strategy UK for its data analytics supplier SAS, “analyses the living daylights out of everything it does.”

One example: The site is known for its powerful personalisation technology – as soon as the site knows who a visitor is, it makes recommendations based on user behaviour. Where does the analytics come in? Amazon randomly assigns users to see different versions of the site, and then analyses how well each version did. Its SAS analytics tool, says Amazon’s Diana Lyle, senior manager for data mining, “is critical for driving value in terms of determining which innovations we should roll out, and in measuring the return on investment of the innovations.”

Lastminute does something similar with its real-time analytics service from Omniture. “We can set up a control group and send, say, 20% of our customers to a new home page and 80% to the old. Then we can analyse the results and see which works best,” says Lastminute’s Horton.

There are many examples of dramatic successes. Josh James, the CEO of Omniture, cites Bell Canada, which made one change to its website, reducing the number of click throughs from 4 to 3, and increased its business by $600,000. In another example, online ecommerce site eBags.com noticed that, at Christmas, its “gifts for men” section was proving popular – an unexpected anomaly. By bringing that promotion up, from the bottom to the top of the page, business in that area increased by 33%.

The most advanced sites, says Dave Armstrong, marketing manager for Information Builders, even make some of the analytics available to the customers. Amazon does this, as does, now, the BBC on its news site. It is building a live stats portal, enabling visitors to see which stories are most popular at different times of day and in different parts of the world. “It’s an anorak’s delight and compelling for anybody who also wants to know what’s going on around the world,” Pete Clifton of BBC News Interactive told a recent conference.

All of this innovation pays off handsomely, according to Neil Mason, principal consultant with Applied Insights, an analytics consultancy. Citing research by online researchers Jupiter, he said that the leading group of analytics users – the so-called strategists – had a conversion rate of 7.3%, whereas the main pack of web operators had a rate of 3% – less than half. That amounts to tens of millions of pounds in sales at the larger companies. Those strategists use analytics more, use complex segmentation more, and a third are planning to hire more specialist staff. Jupiter says only 21% of online retailers can be called “strategists” – but of the remaining 79%, are are planning to invest more in analytics products, expertise and people.