Storing data is costly and it is going to get more expensive. The volume of data – emails, documents, records and so on – that organisations are generating and storing is already rising exponentially. But technological advances in areas such as radio frequency identification (RFID) and IP telephony are about to accelerate that even further.

Little wonder then that storage industry market watchers are confidently predicting that the volumes of data stored will continue to double every year for the foreseeable future. And the impact of that on budgets is already being seen. According to IT analysts at Forrester Research, 35% of European businesses plan for increased spending on storage in 2005 even as overall IT expenditure is stuck in the low single digits.

Without some mechanism of control, the rising tide of data threatens to engulf the IT department. And while legislation and regulation is often regarded as an unwelcome imposition of bureaucracy by businesses, the introduction of rules governing corporate data can actually help organisations impose order on corporate data.

A cursory examination of any cross section of corporate data will quickly establish one simple truth: not all data is created equal. Certain elements of data, say sales figures, may be of vital importance to corporate wellbeing; some data sets, like customer details, are in constant use; others, such as customer emails, may contain interesting and important details but will rarely be called upon.

Through examining the different sets of regulations, organisations are able to classify their data, providing a mechanism to establish what corporate value to place on any given data set.

The Sarbanes-Oxley regulations, which affect businesses with US operations, lay down clear principles governing data integrity. Businesses are expected to demonstrate the veracity of their financial accounts, ensuring that sales figures receive primary importance.

Once an organisation has a clear picture of the value of its various data sets, it can look at how that data is stored. As Bob Passmore of analyst group Gartner explains, the cost of storing data in different media can differ widely.

"If you look at the costs of 1 gigabyte of stored data, typically midrange systems are about 50% of the cost of enterprise systems; ATA disk is around 20-25%; and tape library is 2% to 3%. That ratio has been fairly stable since 1975," Passmore says.

To achieve the most cost-effective method of storing data, IT vendors have been keen to push the concept of information lifecycle management (ILM). Essentially, this is a strategy for managing information assets. That data which is of greatest importance and which needs to be accessed most frequently is placed in the most reliable and available storage device. Data is moved further down the storage hierarchy as its value diminishes.

For ILM to work, organisations must also look to develop a policy-driven, automated approach to data management. Policies are built on a number of criteria, including application performance: where businesses need applications to be responsive, data must be readily available.

Having identified criteria for establishing the value of data, regulations can then help organisations decide how long to store it. For example, the Data Protection Act dictates that personal information is not stored for longer than is necessary for business purposes – in effect imposing a requirement on organisations to delete data.

The Act does not define the length of time for retaining data. But in establishing the principle that data is only kept for a limited length of time, it allows organisations to implement policies that govern when data is deleted – freeing it from at least some of the cost of storage.