Much of the focus at Information Age’s Business Intelligence 08 conference was on how to ensure the success of BI projects. Anthony Howcroft, general manager for data warehouse appliance maker DATAllegro, took a different subject: how to cope with that success when it arrives.

“Like a presidential campaign, the real work on a BI project starts when you win,” explains Howcroft.

As far as the technical infrastructure for BI is concerned, most IT departments only think as far as the initial deployment when provisioning resources. This means that if a BI implementation is a runaway success, they are often caught short.

“The challenge is not making a BI tool work in the 50-user pilot,” Howcroft says, “but keeping it working when those 50 users tell their friends how great it is, and the project takes off.”

The overnight success of BI projects is just one of many factors that can place data architectures under immense strain, he adds.

Another is the growing hunger among executives for more complex reports that analyse an increasing amount of data – such as the so-called ‘weekend report’ that takes two days to run. Existing reporting tools are struggling to keep up with that appetite, says Howcroft.

Moreover, the amount of data that companies are required to hold onto for regulatory or business purposes is growing dramatically, and many businesses are finding their existing infrastructures are insufficient to cope with these requirements.

Howcroft argues that each of these difficulties reflects an underlying shortcoming common among existing data architectures: an inability to support growth.

“People are very good at building clever things,” explains Howcroft. “But building things that can scale is far more challenging.”

The value proposition of a data warehouse appliance, he says, is that the tools have been engineered and tuned specifically for scalability.

Size matters

Remove these scalability issues, says Howcroft, and the benefits can be quite staggering. To illustrate this point, he gives the example of one DATAllegro customer, a US-based telecommunications provider.

The company would use its call data records – the information that describes every single call made over its network – to bill customers. But occasionally it would run an anti-fraud query on this data. This would last many hours, during which time performance on the billing systems would grind to a halt.

The telco invested in a 90-terabyte data warehouse from DATAllegro where all the call data would be staged. Not only did this alleviate the strain on the billing systems when running anti-fraud reports, it also greatly accelerated those reports: what would previously have taken a day now takes a matter of minutes.

With its ability to query the data much faster, the company began to find examples where it had been over-billed by rival telecommunications carriers. Within one month the company had filed a number of claims against competitors, and in its first case won more than the price of the data warehouse.

But, Howcroft admits, selecting the right data warehousing solution is not an easy task. The technical complexity is exacerbated by the fact that many new entrants are joining what was once thought of is a mature market.

“If I were an end user in the data warehousing space,” he says, “I would find it very difficult to choose the right technology.”

He recommends that, rather than sending vendors requests for information (RFIs), potential buyers should start a dialogue with vendors, as it is often the questions they don’t yet know to ask that have the greatest bearing on success.

But by taking care to select the right data warehouse, Howcroft concludes, organisations can make sure that even the most successful BI projects succeed.

Back to the Business Intelligence 08 contents page