For one, the sheer volume of data available for analysis is demanding new approaches. "The figures seem crazy," he says. "The storage of all types of data is growing at unprecedented rates."
That means, say analysts at IT sector adviser Gartner, that by 2012 enterprises will be working with 30 times more data than they did in 2002. And certain emerging technologies may spur even faster growth: companies such as Wal-Mart, the world's largest retailer, are planning to use radio frequency ID (RFID) tags on each and every item that passes through its supply chain. If the data held on those chips was collected, it would total 7.7 terabytes a day, says Gallagher.
And that is just the structured data – the data held in tabular format that has been the historic focus of business analytics. IBM researchers estimate that only 15% of the world's enterprise data is actually stored in a structured format; the rest – comprising word files, emails, scanned documents, voice data and other content – is held in unstructured formats, stored across a multitude of dispersed and incompatible systems.
"BI today only really tackles the area of structured data," says Gallagher, even though most of the 800MB of data that analysts at Ovum reckon is generated by the average corporate employee each year is held in an unstructured form.
Gallagher believes that as part of its evolution BI is going to have to combine the analysis of structured and unstructured data. To date, though, "no vendor has really cracked the unstructured data analysis problem."
He estimates that tools for tackling a mixed data environment are still about two years away. There are many areas where the inability to handle both structured and unstructured data impedes organisations, hitting the bottom line – or putting senior management outside of the law.
Pain points include invoice processing; shipping and receiving ; and product design processes. But the clearest example is in dealing with regulatory obligations. "As organisations, we now have lots more regulatory requests for information – whether it is as a result of Sarbanes-Oxley, Financial Services Authority rules, Basel II or another set of regulations," says Gallagher.
The questions such authorities ask typically require querying and analysing information from multiple sources, he says, "and the BI solutions we build in the future will have to have that ability."
"Today changing business conditions and increased regulatory requirements mean the need to perform integrated analysis of structured and unstructured data is becoming more common – and will become a necessity," says Gallagher. But that takes analytics beyond BI, he says. The tools, techniques and strategies behind that wider analysis he defines as 'information management', a superset of BI, content management and search.
Unless organisations establish an information management strategy, the vast majority will leave large quantities of their data unmanaged and abandoned. In a move designed to capitalise on those developments, Accenture is investing $100 million over three years to develop an Information Management Services practice with the aim of helping clients to "harness, view, manage, analyse and store data, text and other information to improve decision making, financial and operational management and customer service."
That is a hefty bet, but as analysts at Gartner point out, the re-alignment around structured and unstructured data is already under way.
About Steve Gallagher
Head of the BI practice in Europe at IT services consultancy Accenture, Steve is a 20-year veteran of BI. In previous roles, he led IBM’s data warehouse services business, built a similar operation at consultancy KPMG and worked at several BI sfotware vendors. He is now involved in creating Accenture’s new Information Management practice.
