How to transition from data hoarder to data factory

There is nothing positive associated with the term ‘hoarders’. It invokes images of houses jammed to the rafters with stuff their owners haven’t used in years.

More recently, the word has been in play due to a cultural phenomenon named Marie Kondo. This 30-year old Japanese woman has sparked an international movement to ‘de-clutter’ our lives. Her approach to every owned possession is that you take it out and – unless it ‘sparks joy’ (tokimeku) – throw it out.  

Most enterprises these days are data hoarders. Living in the era of big data, with the Internet of Things on the horizon combined with storage being as cheap as it is, IT professionals realise that data is coming at them at a rate too fast to manage, process and analyse effectively. So, instead, they hoard.

The logic behind hoarding is simple and, at first glance, even practical: why throw something out if you might use it later? However, in the world of enterprise data that ‘might’ can be quite costly.

>See also: The big data phenomenon is broken: 5 tips for doing analytics the right way

The CGOC (Compliance, Governance and Oversight Council), a forum of about 2,300 legal, IT, records and information management professionals from business and government agencies, estimates information hoarding costs enterprises thousands or even millions of dollars annually in superfluous infrastructure and storage costs – not to mention legal fees for reviewing documents that should be trashed.

Hoarding also lowers productivity as workers are tasked with sifting through vast stockpiles of digital data to find information they need.

Technically, data hoarding has a downside as well. As enterprises turn to big data analytics platforms to help manage this data, many of them do not realise that the algorithms on which these platforms rely function more effectively with current, near real-time data. Stale data skews the algorithms towards outdated conclusions, meaning that the most current – and most valuable – data is devalued by hoarding.

How does one make the transition from data hoarder to data factory (someone using their data properly and effectively)? Here are some key components to consider.

1. Usage

Ask CIOs and they will tell you: the most valuable data is not the data produced by your most expensive assets – it is the data that gets used.

Just as closet organisers such as Ms. Kondo have a rule of thumb – if you haven’t worn it in the past year, throw it out – data professionals have to similarly set standards for ‘usage’ and live by them.

2. Silos

This is the ultimate in data hoarding. Many IT professionals, doubting the quality of data from their own systems, squirrel away the most important data in silos. In addition to these silos not being available to other users, they are not visible to most data management tools.

This type of partitioning renders data valueless on the enterprise level. Enterprises have to adopt an integrated strategy that identifies silos and makes them available to all parties.

3. Raw data

Considering the variety of data types out there (250 and counting), it is not surprising that a large amount of an enterprise’s data is not in a form that can be managed, much less utilized by the common employee.

Enterprises need a strategy (and underlying technology) to recognise, collect and manage all of that data. In short, part of that data factory should be a ‘data refinery’.

4. Bad data

It is estimated that, at any moment in time, up to 40% of an enterprise’s IT data is ‘bad’, meaning incomplete, missing or inaccurate. This may be the result of human error or faulty discovery tools, but whatever the source, you need to fix it – and fast.

There’s one other item that enterprises have been hoarding since the dawn of the digital age: tools. With data proliferating faster than enterprises can accommodate, in the absence of a plan and supporting platform, IT teams have had to make due with an assortment of tools.

Most of these tools are ‘one-off’s’, capable of performing one data function only, and usually in a limited manner that can’t handle today’s data volumes and complexity.

The key, as with the data, is to optimise these tools – to ensure you have the right number and type, that they don’t overlap in functionality, that they can talk to each other, and that their data isn’t winding up in one of those silos discussed above.

>See also: How big is big data – and what can I do with it?

What is the solution? For the past few years, enterprises have been building on to their data house as quickly as materials became available – but they have done so haphazardly and without a blueprint.

While it is impractical (not to mention, bad business) to shut down construction, continuing to build a business on such an unstable foundation will inevitably result in the entire structure collapsing.

What is needed is not another ‘re-tooling’ of organisations’ data strategies but a new foundation. While data construction crew continue to handle current data issues, organisations should have a small, specialist crew create a new data strategy, complete with a cohesive design and comprehensive blueprints. Then, get to work.

We have moved beyond the era of tools into the era of platforms. With the right platform, businesses can develop and implement a data strategy that captures, processes and analyses all of their data. It may not cure them of their data hoarding tendencies, but it will alert them to whether an intervention is needed.


Sourced from Gary Oliver, president and CEO, Blazent

Avatar photo

Ben Rossi

Ben was Vitesse Media's editorial director, leading content creation and editorial strategy across all Vitesse products, including its market-leading B2B and consumer magazines, websites, research and...

Related Topics

Data Strategy