In last year’s Information Age Effective IT Survey, conducted on the cusp of 2006/07, the deployment of business intelligence (BI) was rated as one of ten least effective strategies or technologies in use by corporate IT.

It was a slightly misleading result – acceptance levels were still very high – but the finding was not entirely surprising. BI is one of IT’s great applications, widely used and enjoying a period of sustained investment, but even insiders in the BI industry have been saying that BI has not quite lived up the hype that has at times surrounded it. One survey, by analytics tools supplier SAS, found that BI and corporate performance management (CPM) tools rarely provided a return on investment.

Part of the problem may have been the hype itself. BI has, in recent years, produced some spectacular success stories, prompting senior managers to buy into the technologies without sufficient thought or with inflated expectations (BI topped the list of global CIO priorities in several surveys, including one by IT adviser Gartner, in late 2006). But research also suggests a slew of other problems ranging from cultural resistance, poor inter-departmental collaboration, a lack of accountability, suspect data, too many tools and a lack of integration.

Research published by Information Age in 2007, based on a survey of some 700 readers, suggested that many people found BI reports irrelevant and difficult to change, could not be easily customised and were not forward-looking or interactive enough. Ironically, BI reports designed to cut through irrelevant information to reveal insights are often themselves viewed as yet more irrelevant data. “BI is typically about producing tens of thousands of reports that go nowhere and convince no-one,” says former lead Gartner BI specialist Howard Dresner. Even BI’s definition itself creates problems. The insights yielded by a multi-million pound investment in a near real-time warehouse and analytics system are not necessarily available to those who merely use a low-cost reporting tool to spice up their Excel reports.

In spite of this, for BI (used in its widest sense to include query, reporting, analysis, performance management and data warehousing), 2007 was a watershed year that demonstrated a market in rapid and fundamental transition. By accident or design, many of the problems identified by end-users – such as the proliferation of incompatible tools and a lack of integration – will be fixed as a result of rapid supplier consolidation.

Over the past 18 months, for example, Oracle has acquired CPM specialist Hyperion, SAP has bought both Business Objects and Pilot Software, and IBM has snapped up Cognos, which itself had already bought CPM supplier Applix.

These acquisitions all reflect the clear belief that buyers now want fewer, broader and more integrated BI tools. Suppliers argue that BI will increasingly become integrated with many mainstream business applications or become part of a wider information architecture that may include content management systems or unstructured data.

There is also a parallel trend in BI that mimics developments in website analytics systems. Although there are currently no clear leaders, many analysts expect that more BI reporting will move online along with key applications such as customer relationship management. Already, several suppliers offer increasingly sophisticated analytics as part of the Salesforce.com (Force.com) platform.

Real-time driver

The big exception to this wave of supplier consolidation in BI in 2007 was the data warehousing sector, where the growing demands for rapid, real-time data and operational business intelligence have created a surge of interest and high expectations. Using these systems, the results of analytic calculations, based on high-performance databases, are fed back into applications and automated business processes for immediate use. This powerful technology, which can be difficult to set up and maintain, helps to address one of the core issues that has dogged BI – that reports are often delivered too late or in a non-actionable form. Emerging BI-based applications can analyse the data and take action before a member of staff has even looked at it.

This interest in real-time, or operational, BI is part of a wider trend sweeping through the world’s largest companies: data-driven decision-making. Many executives, and indeed shareholders, now expect big decisions to be justified and supported not just by executive experience, intuition or widespread consultation, but also by hard data. Analytics systems are being used to measure the impact of decisions alongside simulations and detailed historical analysis. Companies that use these techniques report outstanding successes but large investments in BI – and in training and expertise – are necessary.

This area has been highly active as suppliers enjoy healthy interest from large retailers, telecoms companies, banks, ecommerce companies and others. Teradata, for example, the long-standing giant of data warehousing, was successfully spun out of parent NCR to list on Wall Street. Young rival Netezza also listed on the Nasdaq market. Meanwhile, HP entered the sector for the first time with its Neoview product. Sector analysts also report a wave of young companies offering specialist appliances that accelerate analytic calculations.

Event-driven responses

The focus in real-time systems is also beginning to spawn an interest in complex event-processing systems, where the insights derived from previous analytical calculations are cast into simple, event-driven responses requiring no detailed analysis.

Leading up to 2007, BI suppliers enjoyed a full two decades of strong growth, offering powerful but often distinct and disparate reporting systems. BI is now entering a new phase. The emphasis will be on fast, integrated systems and automated decision-making. In future years, analytics capabilities will increasingly be built in or offered alongside other basic enterprise applications and platforms.

Further reading

Back to the Effective IT 2008 Report contents page

Empowered decision-making How advanced business reporting is driving performance

Competing on analytics Some large businesses are using analytics to give them competitive edge.

Find more stories in the Business Intelligence Briefing Room.