Information Age: News, analysis & insight for IT & business leaders

That, at least, was the theory. In practice, a number of problems quickly emerged.

First, installing devices to run the intrusion detection monitor proved costly and time consuming, and frequently systems had to go through several months of manual tuning before they could be switched on.

Even after they were up-and-running, the systems were not able to distinguish real attacks as accurately as vendors had promised. In fact, the systems generated such overwhelming volumes of 'false positives' that security staff were at a loss to decide which attacks to investigate.

And hackers frequently took advantage of this weakness by deliberately flooding networks with data they knew would generate false alarms in a bid to disguise their main attacks.

The devices were also useless in the event of a 'day zero' attack, the name given to a new hacking technique as it is unleashed for the first time. This is because intrusion detection systems rely on 'pattern matching', in which incoming data is examined and compared with a signature database of activities that indicate an attack. Until the intrusion detection software vendor produces a 'signature file', the system is oblivious to the threat.

Performance ceiling

There was another problem with signature-based intrusion detection systems. As network speeds were cranked ever higher, and with the number of threats still rising exponentially, signature databases became too bloated to provide timely comparisons. This was despite the introduction of ever-faster devices designed to speed the process of trawling for matches as each data packet passed by.

"[Intrusion detection systems] had trouble reaching gigabit speeds. With a great deal of engineering effort, we might have been able to get the first generation products to run at one gigabit per second (Gbps)," says Rob Clyde, chief technology officer of security giant Symantec. "But we were concerned about how we would take them to the next level of network speeds of up to 10Gbps. And it seemed that all the other signature-based systems were having similar problems," he adds.

Moreover, many vendors' products were renowned for dropping data packets even when they were operating at 60% of their supposed capacity, further undermining their accuracy.

But there was one final shortcoming that could not be overlooked. Even where they could identify an attack, intrusion detection systems by themselves were powerless to do anything to stop it once it was underway. As a result, it might be hours, or even days before security staff realised that something was awry.

"Intrusion detection is just that - detection, and it is not perfect at that," says Forrester Research analyst Michael Rasmussen. For all these reasons, an astonishing three out of every four intrusion detection system deployments are ultimately regarded as having failed, he adds.

Abnormal packet

This is where intrusion prevention comes in.

The idea of introducing features that can actively stop an attack is not new. Some vendors have boasted of such features for two or three years now. However, the results can be unpredictable.

Rasmussen reports incidents where the automatic protection in early intrusion prevention devices "was used against an organisation in order to cause a denial of service or to potentially compromise the tool itself".

But as the technology has matured, and new techniques have been introduced, even sceptics like Tarek Meliti at TDM Group are looking closely at the new generation of products.

While arguably not an entirely new category of security product, intrusion prevention does represent a smarter approach to intrusion detection. As with intrusion detection, prevention really covers two complementary classes of products: network-based and host-based.

Network-based intrusion prevention does not abandon signatures. Instead, the signature database is relegated to the role of a secondary tool for confirmation when other, less resource intensive techniques have indicated the likelihood of an attack.

The first of these techniques is protocol anomaly detection. "It turns out that the vast majority of attacks have characteristics that make the packet look abnormal," says Symantec's Clyde. "So rather than look at specific payloads of the attacks, we are looking at the characteristics of the packet itself," he says.

Part of that involves simply examining incoming packets to make sure that their structure conforms to Internet Engineering Task Force (IETF) RFC specifications. Even this is not always an accurate method because some vendors' products do not strictly conform to the IETF's specifications.

The second main element involves statistical flow analysis to detect possible denial of service attacks - the most common type of attack organisations face thanks to the plethora of simple hacker tools available to the mass of 'script kiddies' on the Internet.

If an attack is identified by any of these methods, the data can then be logged and run through the signature database for confirmation and identification, helping security staff come up with an antidote more quickly.

Host intrusion prevention software, likewise, works at a number of levels.

Like host-based intrusion detection systems, intrusion prevention software constantly monitors the operating system log files for suspicious activity, such as ordinary users conducting activity more commonly associated with systems administrators.

Iain Franklin, Network Associates