Besides the microchip, the database is arguably the defining invention of the last 50 years. And yet, innovations such as in-memory, non-relational and distributed databases continue to create new possibilities for information retrieval and analysis.

But for enterprise infrastructure software vendor TIBCO, the heyday of the database is coming to an end. "We see databases as a necessary evil," says Raj Verma, vice president for worldwide marketing at the company. "You need to store data somewhere."

However, a database is always going to be a record of the past, Verma says. TIBCO’s thesis is that analysing data "at rest" no longer confers competitive advantage.

"Making decisions solely on what’s captured in databases gives you a very thin view of what’s happening in your organisation," Verma says. "You have to marry that historical data with data in motion."

TIBCO made its name in the 1990s with messaging bus software that allowed financial systems to exchange data in real time (the name is a corruption of the Information Bus Company). Alongside the service-oriented architecture and business process management infrastructure software it has built or acquired along the way, TIBCO’s messaging systems still make up much of its revenues.

But according to Verma, the company’s consistent growth of around 20% year-on-year is primarily driven by the tools that allow its customers to make sense of this data in motion.

Perhaps the best example is its complex event processing platform, Business Events, which allows users to spot patterns in the stream of events created by applications and other IT systems.

"An event could be a customer buying a shirt, returning a shirt, making a call to complain about the shirt," explains Verma. "Now, those events viewed in isolation may mean nothing at all. But by correlating those events you can spot patterns, and those patterns may represent an opportunity or a threat."

The applications of CEP are many and varied. Telcos can predict customer churn by analysing call patterns, for example, while retailers can predict when customers are most likely to buy products based on their recent acquisition.

There have even been applications in national security. After 9/11, intelligence agencies found that the terrorist shared similar financial histories: their bank accounts were all linked to PO BOX numbers, not addresses; there were no credits or debits on their accounts; they had no life insurance. Having found these correlations, the CEP software can be programmed to flag up when they emerge again.

TIBCO recently boosted the scope of its pattern recognition capabilities with the acquisition LogLogic, whose technology analyses machine log files in real time to detect security incidents.

"The LogLogic acquisition means that our event correlation becomes much richer because now we can correlate application events with machine events," says Vermer. "That means we get deeper intelligence about what’s going on in the organisation."

This could be a potent combination. Processing the vast quantities of data produced by machines is testing conventional data management techniques to their limit, but according to Philip Howard, research director at analyst company Bloor Research, CEP could be the answer.

"CEP is the solution to many of the issues raised by big data, especially instrumented data arising from sensors, logs, smart meters, RFID, GPS or similar," he wrote recently.

Context is king

Another approach to extracting business insight from information assets is to analyse unstructured data, or content. Verma says this too is going the way of the dinosaur. “Our basic premise is that content is no longer king – context is king,” he says. “Context is going to be the differentiating factor.”

To make the case for the primacy of context over content, he points to US retail giant WalMart and e-commerce powerhouse Amazon.com.
 
“The amount of data WalMart has about its customers would fill 130 libraries of Congress,” he says. “What they do with that data is analyse it, and then send out coupons and marketing material weeks later. I don’t think that is a very effective marketing strategy.

“Compare that to Amazon.com which gives you a recommendation right then, by marrying your recent behaviour with your historical data," he says. "It’s a perfect combination of data at rest and data in motion.”

In fact, Verma has some harsh words for one of the UK’s most prominent content management success stories: Autonomy, which was bought last year by Hewlett-Packard. “Autonomy acquired companies that really have nothing in common, put them all together and sold it for ungodly sums of money.”

It seems inevitable that TIBCO will break the $1 billion revenue barrier in 2012. The market for CEP in the financial services sector alone is set to grow by 30% annually for the next two years, according to specialist research firm Celent, while the application infrastructure and middleware market – TIBCO’s core – is growing by 10% annually, Gartner’s latest research reveals.

Having just raised $500 million in debt, the company is looking to acquire new capabilities in adjacent technology areas. “We will invest that money in broadening and deepening our platform with acquisition,” says Verma.

That should be ample ammunition for TIBCO founder and CEO Vivek Ranadive to realise his vision. “Vivek is trying to create the first enterprise software company for the 21st century, Verma explains. “He firmly believes, from every pore of his body, that databases are done with, and that the enterprise of the future is an event-driven enterprise.”