Many people ask why there doesn’t seem to be an accepted definition for big data in view of the massive press it receives. And given the amount of marketing effort expended by the global IT vendors who are targeting this bandwagon (and the venture-funded startups), they can be forgiven for being confused.
Just as most consumer marketing – curiously – is focused on adolescent buyers with a fraction of the purchasing power of their parents and grandparents, so it is with big data and its promoters.
They shower their traditional customers – the IT departments – with buzzwords, while failing to engage with (or indeed, completely alienating) the business managers who have both the business problems that analytics can solve and direct control over the budgets.
Why is this, and what does it mean for big data adoption?
The targeting of IT departments stems from both laziness and geekiness. Laziness in the sense that, with few exceptions, these large vendors got to be large without needing to address directly the concerns of business managers – the IT departments they deal with have until now performed this translation task for them.
And geekiness in the sense that the smaller specialist vendors are often managed by technologists, and so their market messages – for reasons of background – also tend to focus on ‘speeds and feeds’, rather than on solving practical business problems.
This has negative consequences for big data take-up and risks becoming the sand that makes ‘the new oil’ less attractive.
The massive hype – the promise of big data – that has appeared in the press has served to both raise expectations among business managers, and also to confuse them.
Vendors hoping that the IT department will take the lead in initiating big data projects are deluding themselves. IT departments today are a) far too busy with day-to-day work (BYOD, mobile access, security, etc) to do this, b) too distant from the concerns of functional managers to be able to guide them in selecting promising areas for piloting analytics, and c) often unskilled as data scientists.
So, returning to the question of what the term ‘big data’ actually means, there is no accepted definition – however, it’s not about ‘big’ and it’s not about ‘data’ (mostly).
First, it means collecting a wider range of data than the organisation’s current or traditional analysis requires. So instead of running the same monthly reports on the financial system, for example, or on website stats, the business incorporate data that may have no currently known relationship with how it analyses its data today.
Second, it means deriving new insights by combining this disparate data (that may also be external, such as geospatial or social) in new ways. This is more about ‘data discovery’ than about ‘weekly reporting packages’.
The third part is about more effective visualisation – helping stakeholders absorb, share and exploit insights from new data analyses.
If the first pillar (‘question’) is about helping managers better understand the drivers of their business, the second (‘accelerate’) is about speed and accuracy of business response.
Once the discovery part has yielded some insight, the second aspect of big data is about putting it to work. This might be via on-demand reports that enable managers to make decisions more rapidly, or it might be the codification of a set of rules that are applied to the incoming data to make decisions automatically (aka ‘algorithm’).
The third pillar is ‘transform’, the process by which it is deployed. Many vendors see big data as a technology wave, but it should be identified more as a business transformation wave. This is precisely the reason why there is not mainstream adoption until the business change issues are properly addressed.
Implicit in this third pillar is the need to learn to exploit the data opportunity by experimentation. Related to this is the principle of ‘data as a corporate asset’, and by extension to the level of maturity that each business has achieved. The good news is that as more businesses deploy, there is increasing agreement in what defines each stage of maturity.
It is interesting that the technology business is at last starting to move toward definitions that business people would recognise. Paul Maritz, CEO of Pivotal, recently said: ‘Today, thanks to the technologies known as “big data”, computers can capture things as they are happening and can affect events as the events are unfolding.”
So, contrary to all the media hype, big data is really neither big nor about data – it is about questioning, accelerating and transforming.
Sourced from Mike Fish, director, BigData4Analytics