Email, databases, voice over IP telephone networks – there are no shortage of new data sources for today’s enterprise. But as the volumes of data being created within the business hits unparalleled heights, the technologies for storing that data are beginning to show their age. Worse still, innovative, enterprise-ready alternatives are thin on the ground.

As Fred Moore, Horison Information Strategies, a storage consultancy, points out, today’s storage technologies have been around since the heyday of the mainframe. “We are, without a doubt, in at least a three to four year run of reinvention,” he says.

Such pessimism reflects the fact that current storage technologies have remained locked into two primary mediums: magnetic tape and magnetic disk. In the 50 years since IBM announced the release of the first commercial disk drive – capable of holding 5 megabytes of data – the advances in storage have concentrated primarily on increasing capacity requirements and lowering the cost of storage to customers.

There are few indications that this will change anytime soon. But this should be no cause for alarm, says Gian-Luca Bona, department group manager of science and technology at IBM’s Almaden Research Centre. Hard disk technologies still have “a couple of decades to go” before hitting a technological glass ceiling. “This technology is very effective to store lots of data and we are still retrieving it reasonably fast,” he adds.

Indeed, manufacturers such as Seagate Technology, are using currently available disk technologies to push the boundaries of magnetic recording methods ever further: the latest disk drive to emerge from their research laboratories will give a 3.5 inch enterprise drive a capacity of up to 2.5 terabytes of data, space enough to store almost 42,000 hours of music or 800,000 high resolution photos.

Likewise the proponents for tape insist that there is plenty of room for improvement: researchers at IBM expect to be able to increase the amount of data stored on a square inch of tape by a factor of 15, compared to today’s technology.

The reason for this sustained research into things such as new tape heads to realise these improvements is simple: other future options, such as holographic data storage, cannot currently fulfil the requirements of keeping data reliably, in a medium that allows for near instantaneous retrieval of data, says IBM’s Bona.

But this does not mean that organisations can expect technology half a century old to be the storage technology of the future. “We are at a critical point in the storage industry as data moves from analogue to digital,” says Praveen Asthana, director of storage and networking at Dell. “And as we have to keep more and more information, that is putting a burden on how we keep that data,” he says.

Physical limits

Part of that burden is ensuring that data is not corrupted when it is saved to disk or tape. As the demand for capacity has become ever greater, the bits of data that are laid down horizontally on the surface of the disk have been placed closer to each other, greatly increasing the chance that each bit’s magnetic field will interfere – and change – the magnetic field of its neighbouring bit. The resulting effect, known as the superparamagnetic effect, makes the disk less reliable – and it is a problem that scientists have been trying to solve for well over a decade.

To overcome this effect, storage vendors are switching from horizontal to perpendicular recording methods. Instead of lying flat on the surface, the bits are stored vertically, thereby making it possible to cram more bits into the same space. The result: “great capacity along with excellent performance”, says Liam Rainford, engineering manager for Northern Europe at Seagate.

All the major storage vendors are currently developing perpendicular storage devices, but it could be another four years before these products are commercially available. In the meantime, enterprise data volumes will continue to expand, and that means the superparamagnetic effect could come back to haunt business leaders – in as little as seven to eight years, says Horison’s Moore.

To further increase the density of magnetic storage media, researchers are investigating multi-layered or 3D recording, where information will be saved into several layers of the disk, and not just on the surface of the media as currently happens. This technology is very much in its formative stages, but it raises the possibility of increasing storage densities by five hundred times, with capacity measured in gigabytes per cubic inch instead of the current gigabytes per square inch as is the norm today.

However, efforts are also underway to move storage beyond magnetic media. Scientists have already been able to demonstrate the principles of atomic storage – using individual atoms to represent binary ones or zeroes. This potentially increases storage densities dramatically.

Age-old problems

Yet the infatuation with storage densities overlooks one other vital aspect to storage, namely, disk performance. Progress in disk capacity, has grown at an average annual rate of 60% per year; disk performance – measured in terms of access and retrieval times of data – has only improved by just 5% year-on-year.

This is like building a gigantic water tower to supply a town, without increasing the size of the pipes that will transport the water, says Horison’s Moore. This is a “huge challenge in the eyes of everybody except the six companies that build the disc drives. And they couldn’t care less.”

To date the drive makers have compensated for the relatively minor enhancements to disk performance by adding more disks to the storage array. This proliferation of magnetic storage devices means that it will be a long time before it is replaced, says Claus Egge, programme director of European storage at analyst company IDC. “I think it’s likely that magnetic media will be replaced at some stage, but the interesting stuff remains in the labs.”

Further reading

• More articles from Information Age's Storage Business Briefing.
• For more stories like this, check out the Business Intelligence briefing room.