“We’re not worried, it validates our strategy.” This is how almost all IT suppliers greet the entry of a big predatory gorilla company into their market. And almost every time, the stock price plummets (if they have one), and analysts begin to write their obituaries. Mostly, they are right.
So when Google, the search engine giant, announced it was entering the market for online analytics software back in late 2005, it seemed like the old ritual was being played out again. And when it later said its new product would be given away free, the smiles on the faces of the CEOs and marketing people at the rivals began to look like the first stages of rigor mortis. Commoditisation and consolidation seemed inevitable.
The only thing: it didn’t work out that way. In 2006, the market for web analytics has exploded. All the leading suppliers are reporting strong growth, with enterprise customers ignoring Google’s free offer and committing to projects that will cost $200,000 to $500,000 over the next five years. A small army of new consultants in web analytics is being trained up.
And, as David Pool of the web analytics company Site Intelligence points out, this is customer driven, with the world’s leading online and offline retailers leading the way. “We’re pitching to people with job titles that didn’t exist two years ago.” Sitting alongside the CIOs and IT specialists now are online affiliate managers, online business performance managers, customer insight managers, digital marketing managers, and search engine marketing specialists. At many big retailers, online analytics is falling under the remit of a “multi-channel marketing director” – an executive tasked with bringing online and offline sales and marketing campaigns together.
In some cases, whole teams have been created. Tesco, for example, not only has a “digital insight team” that spends most of its time analysing consumer behaviour on its Tesco.com websites, but has also created a joint venture, Dunhumby, that analyses customer and store card owner’s behaviour. “Everything they [Tesco] do is driven by data,” says Pool, whose company supplies analytics tools to the retailer.
At leading online travel site Lastminute.com, it is the same: “We have six core analysts working all day on site analytics,” says Duncan Horton, marketing director. But Lastminute’s use of its analytics package, from US hosted service provider Omniture, extends far beyond a specialist analyst team: “Our licences run into the hundreds in the UK alone. Omniture is not only our main analytics tool, but also our daily management information system”. As he looks out over the open plan office, he says, he can sees the analytics service on most of the screens he can see.
What is driving all this? Google’s influence, say consultants, owes little to recent “validation” of the market, but is considerable nevertheless. As the power of good search engine rankings became apparent from 2001 onwards, search engine marketing (SEM) businesses sprang up to help businesses build traffic. Some of their methods were questioned, some were underhand, and some irked Google, but companies that had invested heavily in their web activity learned a valuable lesson: there is a science to online marketing, just as there is in, for example, positioning and then laying out a supermarket in order to maximise sales.
“We’re pitching to people with job titles that didn’t exist two years ago: an executive tasked with bringing online and offline sales and marketing campaigns together.”
David Pool, Site Intelligence
They didn’t have to look far to find the tools. Business analytics companies such as SAS Institute and Information Builders, were already advocating the use of their warehousing tools as a platform for analysing both on and offline data. They were joined by specialists such as UK-based Site Intelligence, which uses algorithmic techniques originally developed for molecular modelling to analyse visitor behaviour on big websites; and by dozens of suppliers offering web visitor reporting tools, from the market leader Webtrends, and WebsideStory, through to little known but rapidly growing companies such as Omniture. (See boxes on how web analytics works, and the web analytics market).
Many of these tools have been available for five years or even longer, but only in the past two has the market entered the so-called “tornado” phase. “In the last two years, there has been a revolution in the way businesses approach their web activity. The executives are saying: “This website is costing us a fortune. We need to know what we’re getting out of it,” says Conrad Bennett, European technical services director for WebTrends, the global leader in online analytics software by revenue.
Furthermore, he says, no one is content with analysis alone. “All of this is a complete waste of time unless we can take some action,” says Bennett.
That rush to action has been further inspired by some very high profile successes at some of the largest ecommerce and web operators – the so-called “Gods of ecommerce”. These companies – Amazon, eBay, Yahoo, and in the UK, Tesco, the BBC, Argos, and Lastminute.com – use web analytics extensively.
One of these is Amazon.com, which, says Jonathan Opperman, head of eBusiness strategy UK for its data analytics supplier SAS, “analyses the living daylights out of everything it does.”
One example: The site is known for its powerful personalisation technology – as soon as the site knows who a visitor is, it makes recommendations based on user behaviour. Where does the analytics come in? Amazon randomly assigns users to see different versions of the site, and then analyses how well each version did. Its SAS analytics tool, says Amazon’s Diana Lyle, senior manager for data mining, “is critical for driving value in terms of determining which innovations we should roll out, and in measuring the return on investment of the innovations.”
Lastminute does something similar with its real-time analytics service from Omniture. “We can set up a control group and send, say, 20% of our customers to a new home page and 80% to the old. Then we can analyse the results and see which works best,” says Lastminute’s Horton.
There are many examples of dramatic successes. Josh James, the CEO of Omniture, cites Bell Canada, which made one change to its website, reducing the number of click throughs from 4 to 3, and increased its business by $600,000. In another example, online ecommerce site eBags.com noticed that, at Christmas, its “gifts for men” section was proving popular – an unexpected anomaly. By bringing that promotion up, from the bottom to the top of the page, business in that area increased by 33%.
The most advanced sites, says Dave Armstrong, marketing manager for Information Builders, even make some of the analytics available to the customers. Amazon does this, as does, now, the BBC on its news site. It is building a live stats portal, enabling visitors to see which stories are most popular at different times of day and in different parts of the world. “It’s an anorak’s delight and compelling for anybody who also wants to know what’s going on around the world,” Pete Clifton of BBC News Interactive told a recent conference.
All of this innovation pays off handsomely, according to Neil Mason, principal consultant with Applied Insights, an analytics consultancy. Citing research by online researchers Jupiter, he said that the leading group of analytics users – the so-called strategists – had a conversion rate of 7.3%, whereas the main pack of web operators had a rate of 3% – less than half. That amounts to tens of millions of pounds in sales at the larger companies. Those strategists use analytics more, use complex segmentation more, and a third are planning to hire more specialist staff. Jupiter says only 21% of online retailers can be called “strategists” – but of the remaining 79%, are are planning to invest more in analytics products, expertise and people.
The web analytics market
The web analytics market is growing fast – possibly faster than analysts have forecast. A recent report by IDC, the analyst company, put software (including hosted services) sales at $380 million for 2006, up 20% on 2005. It said that the market will double again in the next four years, to $652 million in 2010.
But the three leading suppliers, Webtrends, Omniture and WebsideStory, have all announced growth of more than 70% in their most recent quarters.
And software is less than half the story. Web analytics consultants are, increasingly, in big demand. Most software companies and some systems integrators are rapidly building web analytics consulting practices.
The web analytics software market is highly fragmented. The top five vendors only account for about half the market, with some 20 or 30 other suppliers recording multi-million pound revenues. A shake-out is widely viewed as imminent and inevitable. The most likely casualties will be those with simple reporting products, while those that offer specialist business intelligence (BI) type drill-downs and analysis will most likely stand apart from the fray, at least initially.
There is a huge debate and much division about the value use of real time, “aggregated” analytics, which is fast, simple, and relies on using pre-determined reports; or, alternatively, the use of deep, data warehousing type analytics, which tend not to be real time, and which can be more difficult to use, but yield greater insights and can be more usefully linked to offline data. Suppliers in the former category include Webtrends, Omniture, Google, and WebsideStory; in the latter, SAS, Information Builders and Site Intelligence.
Hosted web analytics is proving popular – partly because it is technically as easy to set up as a hosted service as it is to run it in-house. In most cases, much of the data is anonymous, overcoming at least some privacy concerns. According to suppliers and using analysts’ figures, nearly two-thirds of new business in web analytics is believed to be delivered as a service.
Two big companies will play a big role in this shake out. Google is offering Google Analytics free to any customer who signs up for an AdWords account, and there is widespread agreement it is a powerful and competitive product. And Microsoft, which acquired Deep Metrix in June 2006, is about to make its long overdue entrance into the low to mid-tier of the market.
How Web analytics works
WEB analytics is, according to the Web Analytics Association, “The measurement, collection, analysis and reporting of Internet data for the purposes of understanding and optimising web usage.”
That definition doesn’t begin to hint at the many controversies surrounding the collection of “Internet data”, nor at the powerful commercial reasons why web analytics is such a big market, and such a hot topic today, some ten years after the first products appeared (see the web analytics market box).
When the first ecommerce web server products were introduced in the late 1990s, web analytics software followed shortly after. The web server’s log files recorded who visited what pages, the IP addresses of visitors, time spent viewing and various other data. Extracting and aggregating that data and putting it into a higher level, more accessible format that gave an idea of visitor behaviour was straight forward.
Today’s tools are more like platforms. They are more accurate and resilient, there are many more reports possible, they incorporate search marketing and email integration, but the fundamentals haven’t changed that much since. While some low-end web analytics packages simply show all this data in real time, with a few reports showing daily, weekly or monthly activity, others take a record of every visit and every click made, and export it into a database where it is deeply analysed.
Most leading web analytics products now use page tagging rather than logs. This is largely because caching – which is very widely used to enhance performance – can prevent the actual page being loaded, and therefore tracked. The tags, which must be coded into every measured page, report on visits even when held in a cache. In addition, a lot of other data, such as browser used, or screen size, or other page activities, can be logged.
Using all this data, strategists are able to learn about which pages work best, which campaigns work, and where any web based process can fail. It can yield huge commercial benefits.
But tracking individual users is problematic and controversial. IP addresses don’t always identify an individual user, while only a small minority of visitors to most websites are actually identifiable by name or personal ID. That requires cookies – the small pieces of code that identify a user’s machine to a web site. Many analytics companies have issued their own “third party” cookies, since they often also place the page tags; but these cookies are not trusted on privacy grounds, because they can theoretically track users as they go from one site to the next. So they are widely blocked or deleted.
Although first party cookies – issued by the website operator – are now more commonly used, problems remain. If they are blocked or deleted, it is difficult to make a link between repeat visits. The same applies if a user visits from different machines – or if two or more visitors share a cookied machine.
Many big marketing decisions are often taken on the basis of web analytics – but inaccuracy is still a major and understated problem.