Business intelligence is hard to get right. According to analyst company Gartner, the chance of failure for any given data warehouse project is 50%. And that figure has not improved in the past five years.

So why do so many projects fail? Back in 2003, Gartner believed that the problem was user resistance to adoption. But from 2005 until today, it has identified data quality as the primary culprit.

Roland Spiers, managing director of BI and integration consultancy Clarity Integration, believes that it is all too easy to blame data quality – in truth, the problem is not even as simple as that.

Just look at the many different ways in which BI projects can end unhappily, he says: they can be delivered late; they might deliver much less functionality than originally planned; they can lose the confidence of users because of the data used; they may result in simply the wrong solution. They might even be abandoned entirely.

To catalogue the many ways that BI projects fall down, Spiers outlines what he believes to be the ideal progression of such a project. It begins with setting the focus and scope of the project. This impacts what Spiers describes as the ‘information’ design – what business information is needed and by whom – that will inform the data quality requirements.

Only once these elements are all outlined is the project ready to move on to building, delivering and then testing the BI system.

Broadly speaking, explains Spiers, there are three ways in which BI projects deviate from this ideal path. Firstly, the project is dominated by the business. In this scenario, having laid out its requirements, the business will not make compromises based on feedback from the IT department.

This means that when the IT department runs up against problems, it will use its own techniques to work around them, and the solution will drift away from the requirements, says Spiers.

In the second scenario, the BI project is dominated by IT. Again, this will result in the project drifting away from requirements. “If you don’t keep the users on board from day one,” Spiers explains, “then they won’t be on board when you implement the system.”

In the third scenario, the balance of input between business and IT is appropriate, but execution is poor. While there may be some process to find the precise requirements of business and IT, communication can nevertheless break down and the precise requirements can be lost in the ensuing complexity – again resulting in a system that doesn’t match the users’ requirements.

Tools of the trade

In each of these scenarios, says Spiers, requirements management tools can reduce the risk of project failure.

These provide a platform for collaboration between business and IT, and the benefits are numerous.

By providing a shared repository for all discussion on the BI project, these tools can provide both business and IT with a common language with which to describe the various elements of a BI project.

What’s more, they provide an audit history of the project. This is not only useful for keeping track of progress, but it also helps greatly when key personnel leave the project.

“Many of the BI projects that succeed involve domain experts, who are difficult to come by,” explains Spiers. “Anything that allows you to capture their expertise while they are working for you is going to greatly improve the chances of success.”

According to IT market research firm IDC, requirements management tools can improve productivity in any kind of application development project by 15%. Spiers believes that that can be even greater in the BI domain, given the ample room for improvement.