When it comes to big data projects, measuring progress at every step is an essential component to success. While this should be common sense, there’s still a long way to go with regard to knowledge of best practices for big data project success.
When organisations don’t measure their results, they might go on for months focusing the project on the wrong products, location or people.
Without proper intervention to ‘turn the ship around’, users can become distrusting of results, leading to low levels of usage and acceptance, and ultimately a long tail where results are used less and less.
>See also: How big is big data – and what can I do with it?
Missteps like these translate to lost time and resources for a big data project, or even worse a loss of money for the company.
When a big data project is measured correctly, a company can see big gains through the improvement of a project or solution. Here are three steps for ensuring big data success.
1. Define success
It sounds simple, but identifying what the business wants the big data project to achieve and how to measure achievement is a key first step in any big data project.
For example, consider a project that has the aim of optimising prices on products. Do you want to maximise total company revenue, which is appropriate for market share gain, or profits? And if profits, do you want to also measure long-term customer retention? How will retention be measured? And how will you define project success if short-term profits increase at the expense of long-term loyalty?
When you consider multiple performance indicators, success becomes even harder to nail down. Imagine a grocer of short-life merchandise that wants to optimise their in-store inventory.
The grocer determines that they want to evaluate success based on increasing on-shelf availability, reducing inventory, and reducing spoilage. All three are worthy goals.
Before this inventory optimisation project starts, the grocer needs to identify clear guidelines for what constitutes success when these metrics do not all move in the desired direction. Without this, measurement is simply not effective.
Once the KPIs are identified, the next key step for ensuring success is leveraging a control.
2. Identify your control and don’t lose it
Whenever you’re using data to inform a set of decisions, it’s critical to evaluate if the decisions being made are effective.
Measuring the effectiveness of big data projects means looking at the outcome under two conditions: one where the insight and recommendations from the project are used (test), and one where they are not (control), with all else equal.
Having all else equal is a critical consideration in any test and control. It is imperative to neutralise factors that influence the outcomes in test/control, apart from the fact that one uses the recommendations of the project and the other does not.
Sometimes this requires simulating one of those two states based on what you think the outcome would have been – while other times, you can create a control group by looking at time, location or product dimension where the big data project hasn’t yet had an impact, and then look at how your test and control groups are behaving.
In either scenario, having a control is a critical standard for ensuring the measurement approach is successful.
One common problem that successful projects run into is when the results are so effective that the big data project is used to drive all decisions in all states (time, location, product, etc.).
Once this happens, you have effectively lost control. The best you can usually do in this case is compare the big data project at one point in time to itself at another point – last year’s test becomes today’s control.
3. Engage the end user
Once you’ve identified the KPIs and the control, two key pieces of the big data project are set. But what about the users of the project?
When implementing a new measurement process, one of the most common pain points is a disconnect between the business decision maker and the end user.
Gaining buy-in from the people making use of recommendations coming out of the project is critical – without this buy-in, measurement of the solution may ultimately be obscured by low levels of usage.
One recommendation to ease the tension is to implement specific measures of project success that impact the day-to-day activities of the end user.
For example, how much faster can they accomplish existing tasks using the recommendations of the project? If the big data project has an interface, what is the user experience like? Do they enjoy doing their job more when making use of the insights from the project?
If users see that the big data project is tracking overall company success as well as their satisfaction, they will be more active participants.
While this may take more time upfront, ultimately this will mean better measurement for how well (or poorly) the project is going, allowing issues to be dealt with more readily.
>See also: Top 8 trends for big data
From there, setting up a test and control group that can be monitored for the KPIs of interest, where the test group is impacted by the project and the control is not, is a good way to ensure you’re staying on track.
As an extra layer of effectiveness, identifying one person that is ultimately responsible for reporting on test vs. control KPIs – via an executive dashboard/summary that circulates to key individuals on a regular interval – can cut down on confusion and disagreement on the team.
Despite previous challenges, technology has come a long way to make measurement a seamless and essential part of any big data initiative.
In the coming years, measurement is sure to come along even further, and to be automated in all big data solutions. In the meantime, by identifying the KPIs, creating a control and engaging the end user, big data teams are laying a solid foundation to ensure project success.
Now and as big data becomes even more engrained in corporate culture, it is imperative that measurement is top of mind for the entire business.
Sourced from Ziad Nejmeldeen, chief scientist, Infor
