New Elastic tool trains generative AI with proprietary data

The new Elasticsearch Relevance Engine (ESRE) allows businesses to build bespoke generative AI apps using private enterprise data

Unlike GPT-4 and other widely used generative AI large language models (LLMs), which are trained on publicly available data sources, the newly launched Elasticsearch tool looks to help organisations take advantage of all in-house data, while optimising infrastructure and using talent resources.

Using a relevance engine like ESRE, businesses can build custom generative AI apps, without needing to compromise on size and cost of running LLMs.

Search software provider Elastic has been investing in unified APIs for vector search, as well as hybrid search capabilities and a transformer model small enough to fit on a laptop’s memory.

The ability to “bring your own” transformer model and integrate with third-party transformer models will allow Elastic business customers to create secure deployments that leverage generative AI on specific business data.


What generative AI means for business analyticsJim Goodnight, founder and CEO of SAS Institute, tells Information Age his thoughts on the impact generative AI will have on business analytics.


“Generative AI is a revolutionary moment in technology, and the companies that get it right, fast, are tomorrow’s leaders,” said Ash Kulkarni, CEO of Elastic.

“The Elasticsearch Relevance Engine is available today, and we’ve already done the hard work of making it easier for companies to do generative AI right.” 

Product trials and collaborations

The product of over two years of research and development, ESRE is being trialled by several Elastic customers to augment AI-powered applications.

Legal technology company Relativity, for example, is experimenting with ESRE combined with the Azure OpenAI Service to understand and demonstrate how it might improve the relevance of results within its eDiscovery product, RelativityOne.


How to choose a cloud provider as a law firmHere’s how law firms and legal teams can ensure long-term success from a partnership with a cloud service provider, while staying compliant.


“Ensuring our customers and partners have industry-leading search capabilities is vital to our mission to help them organise data, discover the truth and act on it,” said Chris Brown, chief product officer at Relativity.

“To that end, I’m thrilled about the benefits we can bring to customers through our investments to harness Elasticsearch within RelativityOne. We’re experimenting with ESRE right now, and are excited about its potential to deliver powerful, AI-augmented search results to our customers.”

Julia Liuson, president of Microsoft’s Developer Division, commented: “The expertise of Elastic in enterprise search is evident in the way the company thinks about how to incorporate generative AI into it.

“We’re working closely together to help companies leverage the power of Azure OpenAI and ChatGPT over all of an organisation’s proprietary data in Elasticsearch to help our joint customers deliver a richer and more seamless experience.”

More information on the new Elasticsearch Relevance Engine (ESRE), available today, can be found here.

Related:

70% of businesses currently exploring generative AI innovationA Gartner poll has revealed that 70 per cent of organisations are currently in ‘exploration mode’ when it comes to generative AI innovation.

Avatar photo

Aaron Hurst

Aaron Hurst is Information Age's senior reporter, providing news and features around the hottest trends across the tech industry.