At a high level, requirements for data storage have not changed much in three decades. Organisations want to be able to store and draw on an exponentially growing pool of information, with greater reliability, greater security and lower costs per unit of information stored –  hardly a trivial challenge either for the customer or for the supplier.

Today’s emerging and evolving storage technologies – virtualisation, information lifecycle management, multi-gigabit data transfer, SATA disks, storage encryption and plenty of others – all play a significant role in fulfilling those expectations. But the matrix of choice for customers can be bewildering, says Marcus Schneider, director of storage product marketing at computer giant Fujitsu Siemens – not least of all because most current environments are far from vanilla.

Large and mid-sized companies typically support a blend of heterogeneous storage environments – some configured for efficiency in storage area networks (SAN), some directly attached to server networks for low cost and flexibility, some simply internal disk drives within a server chassis. Even in situations where organisations have established a strategic push towards SANs, that network is often not a pool of re-provisonable storage but a silo focused on serving a single software stack such as SAP or Oracle applications.

In plenty of cases, that makes practical sense, but it does not always serve the goal of getting more out of the existing – and evolving – storage infrastructure.

Storage fiefdoms

At banking giant HBOS, for example, most direct-attached storage is located in branch offices, where managers – each with their own capital budgets – can purchase devices without the knowledge of the central IT department, says Bob Sibley, infrastructure architect at HBOS.

Trying to convince these budget-holders of the ‘greater good’ of shared storage is not so easy. As Sibley says, the notion of giving up something that is ‘owned’ and controlled in favour of something that is shared goes against their feudal instincts.

Focusing on the devices is perhaps the wrong approach. Instead, argue analysts and most storage vendors, IT decision-makers can only get a handle on the storage challenge by focusing on information needs. They need to assess the importance of different information to users, and to provision resources, accessibility, security and other factors accordingly – from real-time access to critical business data through to archived data held on vaulted tapes or data that is deleted.

That philosophy of information lifecycle management (ILM), of course, calls for prioritisation: an ability to classify – and reclassify – data using rules and policies that govern the storage of the data depending on its changing value to the business. And to enable that data migration, organisations will look to an intelligence layer that sits above the storage network, says Dave Goulden, chief financial officer of EMC.

At its simplest level,  ILM may involve the introduction of a classification scheme based on the date when the data was created. For example, data which has not been touched for six months might be moved from ‘live’ media to archived media; data that has not been accessed for 10 years might be deleted automatically.

But this is just a starting point. These time-based approaches take little account of such subtleties as regulations that mandate that a record must be kept for a set time-period. A more progressive approach to ILM is only gained through a clear understanding of the data currently held, how it was created, by whom and the processes that mark its changing value.

No one pursuing an information genome analysis of that scale does so without significant benefits in mind.  For example, at many of today’s businesses the luxury of having an overnight back-up window has long gone. Globalisation and the Internet have turned many businesses into 24/7 operations, making it is essential that back-up can be conducted without taking transactional systems down in what some companies term ‘continuous data protection’.

In theory, the ability to classify data could lead to a tiered back-up process, so that the most business-critical data is backed up continuously, while less dynamic, less vital data is subject to more infrequent snapshots. But data classification programmes are resource intensive, and time consuming – it is not uncommon for them to last between 18 months and two years, says Macarthur. “You have to be careful that you don’t end up trying to boil the ocean.”

Another weakness of this data classification approach is that few line-of-business managers want to engage in the process, says HBOS’s Sibley. “It’s really hard to get users to say ‘this data isn’t particularly important, and should be moved to cheaper storage’. They all want to have the gold standard for back-up.”

Virtual success

Ultimately, new technologies are likely to make it easier to persuade managers to engage with a tiered storage/back-up process, as they introduce enterprise-class performance at a much lower price, says Praveen Asthana, director of storage and networking at computer maker Dell.

Top of his list of technology changes will involve the use low-cost SATA disk drives within standard storage arrays and the widespread adoption of serial-attached iSCSI, the Internet Protocol-based storage networking standard which promises fibre channel levels of performance over an IP network at much lower cost

However, radical changes to storage infrastructure are a long time in gestation, warns Asthana: “Storage buyers are typically very conservative. Their job can depend on where they put your data, so they’re not eager to just jump on the next hot new technology.”

Such conservatism has created silos of legacy systems within current infrastructures as much of today’s storage decisions still centre around tactical acquisitions – often targeted at specific applications – producing a highly complex, heterogeneous environment, much of which does not work together.

To help simplify the management of this storage line-up, vendors have come under pressure to develop – and implement – standards for device interoperability. However, as Dell’s Asthana says: “Work on building a consensus [on interoperability] is slow-going.”

One technology that has been anything but sluggish in its acceptance has been virtualisation. Although primarily a way to improve utilisation levels, virtualisation technologies also provide the benefit that they enable different storage networks to be treated as a single pool of storage. Virtualisation is a “key technology” that may ease many of today’s storage heartaches, says Hamish Macarthur, CEO of analyst company Macarthur Stroud. But virtualisation still has some weaknesses, says Jon Toigo, CEO of storage consultancy company, Toigo Partners – what he calls the plastic surgery problem.

“You can take an ugly man and an ugly woman, and superficially make them look beautiful through plastic surgery. But once they breed, it’s genetics: you get an ugly child,” he illustrates, perhaps missing some of the subtleties of Mendelian genetics. It is the same with virtualisation, he adds: Defects in the data structures are just hidden by a virtualisation veneer.

Further reading

• More articles from Information Age's Storage Business Briefing.
• For more stories like this, check out the Business Intelligence briefing room.