Information Age: News, analysis & insight for IT & business leaders

A more balanced view comes from Axel Goris, a BI specialist at management advisory PA Consulting. “For companies, it’s really a trade-off between the investment they’d be required to make and the business benefit they’d gain from it,” he says.

His colleague, Paul Craig, a database architecture specialist, takes up the theme: “Most business applications do not require the kind of super-performance promised by IMDBs. For those applications, it would be like buying a car that can travel at 150 miles per hour, when you only ever drive at 50 miles per hour.”  

However, there are some areas where PA Consulting is seeing some traction for IMDBs, Craig adds, such as e-commerce applications that have to contend with huge volumes of ‘look-ups’. 

“Take, for example, a rail enquiries site, which is expected to deliver data about train times, stations, routes and so forth in split-second response times,” he says. “Increased traffic to the website can be a nightmare for the company that runs it and any delay can easily result in lost revenue, so a massive increase in performance, provided by an in-memory database, could really help here.”

The same rules apply, he says, to social media applications such as Twitter or Facebook, which dynamically compile content based on user profiles. According to a recent paper from the Department of Computer Science at California’s Stanford University, web companies like these are leading the in-memory database charge.

“In recent years, there has been a surge in the use of DRAM [dynamic random access memory], driven by the performance requirements of large-scale web applications,” the paper reads. “For example, both Google and Yahoo! store their search indices entirely in DRAM.”  

Leading database pioneer Michael Stonebraker also has his eye firmly on fast-growing Web 2.0 companies, as well as massively multiplayer online (MMO) gaming environments, as targets for his recently commercialised in-memory OLTP database, VoltDB (see box-out, Dancing with Elephants).

Stonebraker also sees great potential for the database in underpinning high-volume systems in the financial services industry: VoltDB’s leading investor and inaugural customer is Chicago-based Global Electronic Trading Company. “Anyone that really cares about performance in high-volume transaction environments is starting to weigh up IMDBs,” he contends.  

But are disk-based databases really destined for extinction? One man helping them on their way is Winfried Wilcke, a database research scientist based at IBM’s Almaden Research Centre in Silicon Valley.

Wilcke and his team have recently been focusing their efforts on what they call ‘storage-class memory’ (SCM), in which spinning disks are entirely replaced with solid-state, non-volatile RAM that offers very low latencies (“tens to hundreds of nanoseconds”), low cost per bit and high physical durability.

“Using SCM as a disk drive replacement, storage system products will have input/output (I/O) performance that is orders of magnitude better than that of comparable disk-based systems and will require much less space and power in the data centre,” he predicts.   

In time, he adds, SCM will provide an effective storage system that offers between 50 and 1,000 times the I/O performance of disk drive-based systems. Availability of SCM could be as close as 2012, he says, but the modification of software in order to make explicit use of the technology could be further away, perhaps around 2015.

However, Wilcke does not believe that disks will completely disappear any time soon. “I used to think that SCM would replace disks entirely beyond 2020, but now I’m not so sure,” he says. “In 2030, we’ll still see some disks, but they’ll be used almost entirely for archival purposes.”

But, he adds: “In the face of growing data volumes and the need for fast performance, memory has to be the way forward.”