Driving value from data lake and warehouse modernisation

Oliver Lee-Haswell, Senior Manager at Go Reply, spoke to Information Age about how they have been empowering Data Lake and Warehouse Modernisation with their clients

No matter the sector or size of the organisation, keeping a business’s data organised, actionable and compliant is paramount. To achieve this, Data Lakes and Data Warehouses need to grow alongside the business requirements in order to be kept efficient and up to date. Go Reply is a leading Google Cloud Platform Service integrator (SI) that is helping companies that span multiple sectors along this vital journey.

Part of the Reply Group, Go Reply is a Google Cloud Premier Partner focussing on areas to include Cloud Strategy and Migration; Big Data; Machine Learning; and Compliance. With Data Modernisation capabilities in the GCP environment constantly evolving, businesses can become overwhelmed and unsure on not only next steps, but more importantly next steps for them, particularly if they don’t have in-house Google expertise. Companies often need to utilise both Data Lakes and Data Warehouses simultaneously so guidance on how to do this, as well as driving value from both kinds of storage is vital.

The benefits of GCP

When speaking to the Go Reply leadership team they advise that Google Cloud Platform being the hyperscale cloud of choice for these workloads, brings technology around Data Lake, and Data Warehouse efficiency, along with security superior to other market offerings.

“Google has one of the best software-defined networks in cloud offerings, it is also one of
the fastest utilising dedicated high speed privately owned subsea cables for communication. That means we can get the data into the Google environment through a much quicker path which allows the accelerated delivery of results,” said Oliver Lee-Haswell, Senior Manager at Go Reply.

“Another main differentiator is around security; you’ll hear that a lot of Clouds have Security baked in, and of course they do; But with Google, Security is enabled by default. As soon as data transfers into the Google Cloud cloud network, and you start consuming Services on Google Cloud, all information and traffic travels internally to Google’s systems. Google ensures that your traffic stays within their data centres in a secure, highly encrypted way.”

Go Reply helps customers migrate data from Hadoop to Dataproc, and then to BigQuery infrastructure in order to improve scalability while reducing costs. Businesses can automatically increase or decrease servers and clusters depending on their current data needs, meaning they never spend more than required to perform the task at hand, with queries able to be completed at speed during high volume periods.

Boosting Data Lakes with AI and ML

Implementing Artificial Intelligence (AI) and Machine Learning (ML) for managing unstructured data in Data Lakes helps improve the intuition of data processes, maintaining flexibility in the wake of constant change. Automating tasks also reduces strain on data engineers, allowing them to focus more on general value-add projects.

Oliver Lee-Haswell identified the following key use cases that stand out as particularly
valuable to recent Go Reply clients:

  • Yield management: A great example of this is the work Go Reply have done with an airline customer to improve seat pricing in relation to seats occupied, as the company navigated out of the pandemic. The model implemented can calculate ideal seat prices and marketing spend, to help the airline maximise return-on-investment, while also saving on carbon emissions through ceasing to fly out sparsely occupied planes.
  • Contract data ingestion and mapping: A telecoms company used natural language processing, (NLP), capabilities through BigQuery to turn unstructured contract data into structured data that can be traversed and queried more easily. This way, the organisation can more accurately measure not only how much each B2B customer should be charged in line with usage but also the reduction in errors and time savings by staff.
  • Identifying under and over performing retail locations: A leading drinks manufacturer and distributor approached Go Reply with a specific problem. They wanted to identify the gap between underperforming and overperforming sights to give clear insight and understanding of revenue opportunities and how to address them using specific Google tools.

BigQuery and Looker for Data Warehouses

Data Warehouses store structured data, unlike Data Lakes which typically store unstructured data, and are tailored towards meeting long-term business goals, rather than immediate use of assets. By using BigQuery and Looker in the GCP ecosystem, businesses can leverage data intelligence through turning masses of information into actionable insights. Combined, the two products can deliver interactive dashboards and reports for any dataset.

“It always comes back to speed,” Oliver Lee-Haswell explained. “With BigQuery and Looker it’s no longer about how quickly we can transfer the data into the system – this is solved by leveraging the speed and power of Google Networks; it is now about how quickly a data engineer or data scientist can start using that data, it comes down to how efficiently data can be used to drive insights using intuitive tools and often pre-built Google supplied learning models.

“We’re now exploring with our clients how presenting data in a visual form which makes sense to more people is affecting the speed of business change. Data engineers and data scientists can more accurately represent their ideas through interactive dashboards from Looker.

“BigQuery provides fully centralised data, allowing us not only to remove silos, but also mix that data together for rapid development, while ensuring that data sovereignty and authorisation is in place.

“No longer are we waiting for a PDF report to turn up on your desk or in your e-mail inbox every month.”

Emphasising collaboration

As a Premier Google Partner Go Reply prides itself in its collaborative and consultative approach with its clients throughout their data modernisation journeys. After all, it’s never a short-term engagement, but a partnership lasting months, and often years.

“We investigate and discover what our clients are doing today, we work out why their business processes are playing out the way they are and also where they want to be,” said Oliver Lee-Haswell.

The two-way approach Go Reply practices empowers data collaboration to drive insights, helping customers grow as they advance, but with the future of the business always in mind.

This article was written as part of a content campaign with Reply.

Related:

Global public cloud end-user spending to reach nearly $600bn in 2023 — Gartner has projected end-user spending on public cloud services worldwide to grow 20.7 per cent year-on-year to total $591.8bn in 2023.

Best practices for modern enterprise data architecture — Dietmar Rietsch, CEO of Pimcore, identifies best practices for organisations to consider when managing modern enterprise data architecture.