Until recently, being charged with leading a data quality project was “a sure sign that the boss didn’t like you,” says Frank Buytendijk, VP of corporate strategy at business intelligence vendor Hyperion: the chances of success were so slim, the responsibility was akin to notification that a P45 was being readied.

Not so today, says Buytendijk. The importance of data quality is being embraced at the highest echelons of the business – thanks in part to the punitive regulations that now govern corporate finances. “It is one of the few good things to have come out of [the] Sarbanes-Oxley provisions,” he adds.

Now senior executives are aware they may be held personally responsible for the accuracy of corporate accounts, data quality is on the radar. While data quality is not a new issue, it has become an intractable problem through the proliferation of data sources. And while data cleansing technologies have been available, more mundane ‘people’ issues can still wreck data quality initiatives.

Quality control

Businesses are reliant on the data they hold. As the Lord Renwick, chairman of IT lobby group Eurim, says: “What drives today’s business is the quality of decisions. These are taken on the basis of the available information, so the more accurate and timely that information is, the better the decisions that are made.”

Inaccuracy and inconsistency are the biggest problems; front line data entry staff are often identified as likely culprits. All too often, the front line staff see little of the impact of poor data quality. Furthermore there has been an organisational unwillingness to deal with data quality issues; a torpor that has seen it passed off to any feasible department – the ‘information’ part of IT has made it seem a logical candidate.

But Trevor Dukes, head of information strategy at WH Smith, is adamant: “IT on its own cannot improve data quality”. Staff at every level must take responsibility for their data. For Dukes, this is not derogation of responsibility from the IT function, but an acceptance that quality must be addressed throughout the organisation; centralised functions such as IT can only provide guidance.

Handing the problem to the IT department, based on a spurious notion that IT should be responsible for data, will not help the organisation improve its decision making, he explains. His department only takes responsibility for data quality once it enters the central data warehouse.

Centralised data quality is also a major issue for Transport for London (TfL), the body that oversees London’s public transport. Various parts of the system are contracted out to third parties, making it “challenge” to integrate performance data, says Olliver Robinson, a senior business analyst at TfL.

TfL’s challenge is to find “types of data that are important” to everyone working in the system, so individuals take responsibility for data quality, says Robinson. This also requires high-level data modelling, to understand the nuances of how the data relates to operations.

The need for such understanding from the upper echelons of the organisation is a problem in itself, says Lord Renwick. “I’ve sat on many company boards over the years. I’ve never met a CEO that understands data quality.”

Further reading in Information Age

Business Intelligence briefing room