Process, technology, and team: the core components of data management

Without the proper technology systems and process flows, data can’t be delivered or analysed. Without an expert team to manage and maintain the setup, backlogs and errors become common.

Before planning out a data management strategy, consider what systems and technologies you need to add; what improvements can be made to existing processes; and what roles will be affected by these changes.
As much as possible, make sure any strategy is going to integrate with your current business processes.

It’s important to take a holistic view of data management. After all, a strategy that doesn’t work for its users will not function effectively for the organisation.

>See also: Cloud data management: data protection

With that in mind, this article will examine each of the three primary non-data components of a successful data management strategy: the technology, the process, and the people.

Identifying the right data systems technology

There is a lot of technology involved in big data, and much of it is in the form of highly specific tools. Most enterprises will need the following types of tech:

Data mining

This isolates specific information from large data sets and transforms it into usable metrics. Some familiar data mining tools are R, SAS, and KXEN.

Automated ETL

The ETL process extracts, transforms, and loads data so it can be used. ETL tools automate this process so that human users do not have to manually request data. Plus, the automated process is more consistent.

Enterprise data warehouse

A centralised data warehouse that can store all of an organisation’s data — and integrate related data from other sources — is an indispensable part of any data management plan. It keeps information accessible and associates various kinds of customer data for a complete view.

>See also: It’s time to show big data more love this Valentine’s Day

Enterprise monitoring

These tools provide a layer of security and quality assurance by monitoring critical environments, diagnosing problems when they arise, and quickly notifying the analytics team.

Business intelligence and reporting

These tools turn processed data into insights tailored to exact roles and users. Data has to go to the right people in the right format for it to be useful.


These combine highly-specific metrics (customer acquisition data, product lifecycle tracking) with intuitive and user-friendly interfaces. They often integrate with non-analytics tools to ensure the best possible user experience.

It’s important not to think of the above technologies as isolated entities; rather, think of them working together as an organised unit.

Next, picture the processes used to link data technologies and people together. While maintaining data quality is essential, are there areas where certain tasks can be consolidated? What tasks cannot be performed together?

Optimising process flow

Data rarely travels in a straight line from its collection point to the end user. Along the way, it passes through various phases, including:

  • Data consolidation.
  • Data assessment.
  • Quality analysis.
  • Data modeling.
  • Data analytics.

At this point, it’s important to consult with company stakeholders. Get their input regarding existing process documentation, methods, and quality controls.

>See also: 9 tactics for getting a firm grip on data

Find out how these might be optimised without sacrificing data quality and integrity. This is when many potential problems can be identified and prevented.
Finally, remember that an intelligent data management strategy is never just about technology and tools. It’s also about people.

Finding the right people

Finding the right people for Big Data implementation means recognising three basic truths:

  • Someone needs to manage Big Data’s adoption and educate its users.
  • Someone needs to be responsible for data governance and quality.
  • Someone needs to keep data secure and compliant.

When a person says someone, they’re speaking of a collective someone. Why? Consider each aspect individually:

Adoption management and education

There will likely be at least a few changes in organisational structure when big data is implemented.

In all but the very smallest of organisations, using just one person to train or help all users with analytics is not a workable solution. Therefore, having a designated team to assist other departments is vital to productivity and data management ROI.

Data governance and quality

Data stewardship is imperative; without it, quality can degrade and the information itself can become inaccurate.

Data validation is usually done both manually and by automation, but even so, someone has to be responsible for maintaining its overall quality.

>See also: Data hoarding creates a digital wasteland

Designating either a team or a point person for data governance can ensure data entry standards are met, that errors are resolved, and that data is being properly validated.

Data security and compliance

Data breaches are every organisation’s nightmare, and governments impose strict rules on keeping customers’ private data private. Meeting these requirements calls for a knowledgeable and dedicated set of experts.

How your organisation fills these roles will be based on its size and needs, but they are an essential part of data management strategy.

Like a three-legged stool, data management strategy needs all of its key components. Remove technology, people, or process, and the entire framework becomes unstable at best.

The primary goal behind implementing big data is to empower an organisation by providing the right people with the right information. A properly planned and executed data management strategy is critical for the successful adoption of big data.


Sourced by Anil Kaul, CEO of Absolutdata

Avatar photo

Nick Ismail

Nick Ismail is a former editor for Information Age (from 2018 to 2022) before moving on to become Global Head of Brand Journalism at HCLTech. He has a particular interest in smart technologies, AI and...

Related Topics