Q&A: the impact of big data on the insurance industry

Dan Keeley, head of data engineering at challenger insurer hubb, spoke to Information Age about the impact that big data has on insurance.

With the amount of personal and financial data that is generated across the insurance sector, it’s vital that companies dealing in the space find efficient, secure ways to drive value from assets. Dan Keeley, head of data engineering at Insurtech company hubb, discussed the impact that big data can have on the insurance industry.

How do you see the growth of big data developing in the next few years – and what will that mean in terms of data gravity for the insurance industry?

The growth here will reflect the expected growth in Insurtech funding, which looks to be wild right now. Anyone not investing in a modern data platform will simply get left behind. The challenge for the Insurtech folk is to produce a product of value which appeals to a significant market. It’s interesting to look at banking and how the open banking standards has led to an explosion in modern apps and functionality for users (as well as a few hiccups along the way!). Ten years ago, we’d never have believed the banks could have pulled this off.

What are the main elements driving the increase in big data in the insurance industry – and why?

Serverless cloud computing is the number 1 technology that will allow you to process data in a scalable and cost-conscious manner. We managed to reduce our monthly spend by 80% purely by going serverless, rearchitecting to use 3 or 4 core services, whilst still providing the same functionality. To make the most of all these services you’ll find that everyone will need to provide an API, and join in to the data ecosystem.

How will an increase in big data affect the insurance industry?

It can only be positive. A modern data platform is going to achieve two key things in insurance:

  • an increase in efficiency, reducing cost, and reducing premiums for the customer;
  • a decrease in risk, again allowing better management of the book, improving margins, which in turn can improve competitiveness in the market.

How can companies effectively aggregate their data to maximum effect, and which technologies are proving most effective?

Edge computing is all about IoT, and the only IoT use case right now is telematics. However, the rate of data produced in telematics is easily handled in any moderate cloud-based stream. Whilst IoT data in a raw form does appear to be of enormous size, in reality what it boils down to is identifying the key data points just before and possibly after a particular event. Aggregation of time based data is very easy too with techniques such as change detection, outlier analysis and so on.

With the implementation of smart cities, are these increased amounts of data collection something we should be cautious about in terms of privacy for individuals and security?

The privacy sector is growing hugely now, with organisations such as Privitar and similar having extensive platforms to handle these issues. By studying and engaging these systems everyone can learn tips and tricks to help keep the data secure, and where necessary anonymised. There’s a catch 22 where we can all benefit from with open data, which is in opening the data you must ensure it is safe to do so. Some companies, such as TfL, have risen to the challenge and produced the data in an effective way, while others have simply backed down and done the bare minimum.

How do you see the future of big data developing in the Insurtech space?

I think the term big data is finished. We’re beyond big data now. For me, big data was a term that was used when legacy systems all got bogged down in the amount of data coming in. This is no longer a problem anymore – modern cloud platforms do not have this issue and are inherently scalable. Therefore, you could argue that big data is now ubiquitous.


Navigating insurance M&A complexity during a pandemic — Adam Miller, IT director at Markerstudy Group, discusses the tech choices made to deliver remote working, whilst supporting M&A in insurance.

Q&A: Hubb COO on shifting towards a metaverse infrastructure in insurance — Following the announcement that challenger insurer hubb is preparing for a shift to the metaverse, the company’s COO Edward Halsey spoke to Information Age about how the metaverse can drive value in the insurance sector.

Avatar photo

Aaron Hurst

Aaron Hurst is Information Age's senior reporter, providing news and features around the hottest trends across the tech industry.