Henry Blodget must wish he had just picked up the phone. Over 1999 and 2000, while bankers at investment bank Merrill Lynch had been talking up the prospects of the high-tech companies they were hoping to bring to Wall Street, the then high-flying Internet stock analyst Blodget and his colleagues were warning the bank’s private investment clients to steer clear of many of the very same companies.

Unfortunately for his employer, Blodget used email to do so. When regulators investigated Merrill Lynch and discovered the emails, some of which described their former recommendations as “a piece of shit”, the bank had to settle with them, eventually paying out a painful $100 million in penalties.

Merrill Lynch might have wished it had wiped the emails from its systems before the legal action, just as others on Wall Street had been doing, but even that would not have helped. Around the same time, six investment banks found themselves in dangerous territory, since they had failed to archive all their emails for the minimum six years required by law. The banks – including Morgan Stanley, Goldman Sachs, Deutsche Bank and others – now face fines imposed by the US Securities and Exchange Commission (SEC) of up to $10 million.

These cases are not unique to the financial services sector. Indeed, IT decision-makers in just about every area are hearing alarm bells ring. What they have realised is that email – the most important business communications tool of the digital era – has become a management minefield with many critical decisions to be made. How much email should be kept and for how long? Does the law allow companies that are not involved in financial services to purge their old emails and avoid the contents becoming exposed during any legal discovery process? Are there mechanisms for selectively retaining only important emails? And how can companies put in place systems to easily retrieve these key documents?

Such questions are often overlooked, simply because of the day-to-day fire-fighting that accompanies most email management. The average email administrator has to struggle with huge volumes of emails, tackle a deluge of spam and virus threats, all while handling user requests for access to old emails.

“Actively managing email is a nightmare,” says Ian Campbell, CIO at UK telecommunications and service provider Energis. “What has changed is the fact that email has become a critical part of business operations – on a par with applications such as enterprise resource planning, sales management and customer relationship management.”

Moreover, users have become email dependent to the point where many cannot function when offline or when performance slows dramatically. That situation means that IT directors like Campbell are now having to submit monthly email reports that measure performance against service level agreements. Meeting those service requirements is certainly not going to get any easier. On an average day in 2002, 31 billion emails were dispatched worldwide, three times the volume sent in 2000, according to market research company IDC.

Managing this growth is placing an incredible strain on IT departments, and is prompting many companies to tackle the problem through a combination of policy and technology. Inevitably, some policies are specific to the demands of particular industries, but there are some fundamental components common to all. In many cases, enterprises established those policies when email management was less demanding and less critical – and have not updated them since. Analysts, such as Jonathan Penn, a director at Giga Information Group, have not been impressed by the email archiving efforts of many. “Most organisations have a very casual approach to email, even in industries where there are stringent regulations pertaining to email,” he says.

There are many companies that do not have policies at all. Maurene Caplan Grey, a specialist in email issues at research group Gartner, says that the vast majority of assignments she takes on come from IT managers who have been asked to put together strategies for email management and are unsure where to start.

Traffic jam

Gareth Richards, a managing partner at German document management software provider Ixos, says the focus for most email archiving policies tends to be on three core areas: performance, knowledge and compliance.

Performance problems with corporate email servers tend to be directly related to the amount of email traffic they have to process. A major problem is that leading email servers, such as Microsoft Exchange and IBM’s Lotus Notes, use a relatively limited database system that can quickly become saturated, says Nigel Dutt, CTO at messaging software specialist KVS.

This means that a company that does not deploy some form of email archiving technology regularly has to buy additional email servers.

The promise made by some of the new breed of archiving software is to offload most of the company’s email database onto a separate server or device. They can also free up additional capacity on an email server by eliminating duplicate messages. Customers can then avoid purchasing additional email servers.

At the US Air Force, for instance, the Air Mobility Command unit was able to reduce the number of Microsoft Exchange servers it used from 30 to 12 by adding an email archiving server from Ixos called eCONserver. Ixos is not alone, with suppliers KVS, OTG, iWitness, MDY, eManage, Legato, StorageTek, Veritas, TrueArc and Gordano all providing similar products.

Many buyers are attracted to such systems because they make it easier to comply with regulations. Central to this is the ability to retrieve emails quickly and efficiently. So archiving products index emails using parameters such as key words, file size or chronology to ensure access is as quick as possible, says Dutt at KVS.

Effective retrieval can also save organisations large sums of money. “Without indexing, it can cost companies hundreds of thousands of dollars to retrieve emails,” says Penn at Giga. In fact, market research company Coalition for Networked Information claims system administrators spend between five and six hours every week recovering archived messages and attachments for users. More complex retrieval searches can take much longer. Before US financial services company Davenport &Co installed an email archiving system, administrators at the company sometimes took as long as two days to find critical emails, reveals Davenport’s network technician Jeff Joyner. As a result, the company decided that it needed a way of accessing old email that was easier to use and more cost-effective, choosing to implement EmailXtender from data storage software company Legato.

In the subsequent six months, Davenport sent, received and archived more than 500,000 email messages, creating an archive of about 33 gigabytes, says Joyner. However, a detailed search of the archive using EmailXtender only takes between 10 and 15 minutes. Providing that retrieval capability also creates access to a knowledge base, even though many users often limit that sharing and duplicate a significant number of emails by setting up personal email folders for local archiving (see box, In practice: Sony UK).

Preventative measures

Companies risk losing vital corporate knowledge if they do not deploy some form of email archiving system. They can also safeguard against deliberate damage. A case in point is DNM Technology. The Irish software consultancy found an unexpected security benefit after it deployed an email server from Gordano, the UK-based messaging software developer. DNM became aware in Spring 2002 that a disgruntled senior sales representative, who realised his days at the company were numbered, had been deleting large volumes of business-critical emails. “These emails included all the emails he had been sending to clients containing quotes for contracts – so if we had not had Gordano’s NTMail software in place, we would have had no record of them,” says David Quirke, IT services director at DNM Technology.

Most email administrators and IT managers are fully aware of the perils of lost email. For many, email archiving is a trade-off between the business need for a robust email system and the pressure on the IT department to reduce management costs, says Mike Coombs, a messaging administrator at CMP Information, the professional media publishing division of United Business Media. “The biggest problem is balancing business requirements, legal requirements and cost. Adding archiving features costs money in terms of hardware, software, deployment and support,” he adds.

Campbell at Energis has a similar view. “We are under tremendous pressure in terms of cost management, but at the same time we are expected to deliver a bullet-proof corporate email system.”

The fact is that companies often do not start to formulate an email strategy until it is too late and the regulators or lawyers are knocking on the door.