Information Age: News, analysis & insight for IT & business leaders

BRINGING information together from a number of disparate sources inevitably leads to data quality issues. The potential damage created by relying on a corporate database that is awash with incomplete, out-of-date, duplicated or redundant records can be huge: a loss of trust with customers and suppliers; the breaking of numerous regulatory compliance laws such as the Data Protection Act or Freedom of Information Act; and the high cost of storing duplicated data and then transferring it to secondary storage devices.

Mike Pratt, data integrity manager for Business Link for London, is familiar with the chaos that bad data can cause. Until recently, a quarter of all records in the company’s customer relationship management system were duplicates.

Business Link for London was created in the 1990s as a portal for entrepreneurs in London. Last year, for example, it provided advice to over 105,000 groups and individuals about starting and growing a new business and it provides an online support service to over 100,000 clients.

Until 2001, it operated through nine distinct agencies across London. However, when it was decided to consolidate these into a single support agency the client records of each unit were simply dumped into a single repository with little or no regard to the consequences of duplication, says Pratt. That problem was exacerbated in June 2003 when the highly flawed database became the main feed for Business Link’s PeopleSoft customer relationship management system.

However, over the last two years Business Link for London has been reducing the duplication, using data quality tools from FirstLogic it has taken the incidence of duplication down from 25% to 2.5%. And by passing data through a validation process before being committed to the database – such as validating postal addresses by cross referencing to post codes, for example – it is aiming to reduce that figure still further this year to between 1.5% and 1%.

The de-duping exercise has resulted in substantial savings, reducing the direct marketing costs alone by up to £100,000. “We are sending the right material to the right people and marketing is putting its spending into the correct areas now,” says Pratt.

Data quality is a growing concern for many businesses, especially as the use of business intelligence (BI) as a strategic decision making tool becomes more prevalent. And the BI vendors are slowly recognising the value of the old adage “garbage in, garbage out” with several key vendors acquiring data quality software companies to improve the offerings. This year, for example, has seen the acquisition of FirstLogic by BI suite vendor Business Objects; Informatica’s purchase of Similarity; and, most recently, Hyperion’s buying of UpStream Software.

Pratt feels that Business Link for London will not see the advantages of FirstLogic’s acquisition by Business Objects in the short term. Indeed, like many organisations, it is still debating the future of its business intelligence strategy, but is swaying towards Microsoft’s SQL Server BI offering.