The business intelligence market is expected to almost treble from $3.6 billion (€3.9bn) in 2000 to $10.1 billion (€11bn) in 2006, according to IDC. Still, with literally hundreds of specialist companies addressing everything from data preparation software to report generation for mobile devices, a shake out is inevitable — and well overdue.

BI ‘can prevent another Enron’

Analysts at Gartner believe the market for BI tools is being given fresh impetus following the collapse of energy trading giant Enron. The market research group has championed the concept of the ‘glass enterprise’ — a transparent organisation, created by BI technology, through which employees, shareholders and, if necessary, financial regulators can peer through to learn about its true condition.

“A lot of executives are worried about what is happening in the fallout from Enron, Marconi and Allied Irish Bank,” Nigel Raynor, Gartner research director, told the analyst group’s recent conference in Florence, Italy. “Nobody wants to be the next Enron.”  

Building the Knowledge Base

There are considerable differences in the functionality and pricing of extraction, transformation and loading (ETL) tools used to populate data warehouses. But all have the same aim: to grab data from application databases, reconcile the differences in the data, and move this into data warehouses for use by query, analysis and reporting tools or analytic applications.

Although the ETL segment has produced no real software stars, huge opportunities still exist for automating this critical function. Today, in 75% of data warehousing projects, organisations get their own programmers to write routines to extract, transform and load data, according to Gartner.

But at the same time, ‘pure play' companies, such as privately held ETI, Ascential and Informatica, are seeing their wares commoditised. The reason: Microsoft, Oracle and other database management systems vendors are bundling ETL tools with their mainstream products, giving them an ever-greater share of the ETL market.

Better choice

Analytic applications are touted as the next generation of BI software. Organisations are under pressure to manage costs better, optimise supply chains and capture trends in customer behaviour. Analysts say that many users want applications that address those specific tasks, rather than have their IT departments build analysis capabilities using generic analysis and reporting tools. At the same time, they are increasingly frustrated by the analytical shortcomings associated with enterprise resource planning, supply chain management and customer relationship management packages.

While earlier analyst expectations of a widespread shift from tools towards applications may have been too aggressive, the analytic applications market is certainly gaining momentum.

For now, BI vendors still tend to target specific segments of analytical applications. Hyperion and Comshare largely focus on financial analytic applications, for example, and there are a host of companies addressing customer relationship management, web and supply chain analytics. But the market is still in its infancy and is highly fragmented.

Not to be left out, the leaders in data analysis tools – Business Objects, Cognos and SAS Institute – are supplying suites of applications. And the major business applications vendors – such as SAP, PeopleSoft and Oracle – are adding analytic application capabilities to many of their modules. Recognising that, analysts at consultancy PricewaterhouseCoopers believe that “data warehousing and BI will increasingly become embedded functions offered within a myriad of packaged corporate applications.”  

Microsoft muscles in

Online analytical processing (OLAP) addresses the limitations of relational databases by enabling users to analyse data across multiple variables.

From nowhere in 1998 (when it acquired the OLAP technology of Israeli start-up Panorama), Microsoft has become the joint market leader in the sector along with Hyperion Solutions. It has achieved that largely by bundling OLAP services with its SQL Server database, a move that has driven down prices for OLAP software. The company also set the trend for large vendors, such as Oracle and IBM, to embed their OLAP technology within their database management system (DBMS) products.

Like those companies, Microsoft also bundles extraction, transformation and loading capabilities with the underlying database technology. But the vendor’s BI ambitions run much deeper.

In 2001, the company added to its BI portfolio with the acquisition of Israel-based Maximal, a developer of an analytic and visual toolset for front-end analysis. The product has since been renamed Data Analyzer. At the other end of the spectrum, Microsoft has progressively increased the BI capabilities of its Excel spreadsheet package. But that still leaves a gap in the middle of its BI line — which could be filled by the acquisition of one of the many financially insecure query, analysis and reporting tools vendors.

Digging into the customer

Demand for customer relationship management (CRM) analytics products will rise at a compound annual growth rate of 27%, from $467.3 million (€506.6m) in 2000 to $1.5 billion (€1.6bn) in 2005, according to market research company IDC.

That kind of optimistic forecast has inspired specialist vendors in data analysis, infrastructure, business applications and financial analytics to rush to market with CRM analytical applications.

All these vendors point out that CRM applications may be attractive in themselves, but it is the analysis of the customer data generated by these systems that provides the real opportunity for users looking for a significant return on investment. That was the driver behind CRM leader Siebel Systems’ decision to acquire analytic expertise with the acquisition of nQuire in October 2001. It now embeds analytics in its applications.

One related but largely separate aspect of CRM analytics is also widely deployed. Since 1997, there has been a steadily growing market for analytics that measure and track the behaviour of visitors to a web site. Specialists in web analytics applications include Accrue, WebTrends and SPSS’s CustomerCentric unit.

Housing the data

Database management systems (DBMS) are the bedrock for creating and managing data warehouses. Data drawn from multiple transactional databases is stored in these warehouses for analysis and reporting. Oracle, (with Oracle 9i), IBM (with DB2 and Informix), Microsoft (with SQL Server) and NCR (with Teradata) dominate this BI platform sector, but upstarts such as Sand Technology and Alterian are generally doing well in niche areas.

Oracle may be the most widely deployed platform by dint of its dominant position in the Unix database market, but NCR’s Teradata division is widely regarded as the technical leader. Teradata boasts a strong, and growing, presence in the high-end, large volume BI database market. Meanwhile, Microsoft’s SQL server is gaining market share in low-end projects.

Q&A

Query and reporting tools are designed to give end-users access to BI.

The market for query and reporting technology is the most fragmented of all the BI sectors. There are hundreds of tools from a myriad of vendors designed to generate reports and highlight trends in, say, sales. Query and reporting tools are still sold separately, but increasingly the leaders in data analysis – Brio, Business Objects, SAS, Cognos and Oracle – supply integrated suites.

The trend toward sharing information with more users within an organisation has spurred these vendors to web-enable their products, allowing wider access to BI through the Internet and intranets. Reporting is seen as the main mechanism for delivering BI to a wider audience.

Currently, however, the strongest demand for reporting tools comes from companies wanting to deliver extranet reporting to provide customers with snapshot reports of their account activity.