Darktrace applies Bayesian theory to cyber security

The career of Dr Mike Lynch, former CEO of Autonomy, has benefited greatly from the work of 18th Century presbyterian minister Thomas Bayes.

As well as a devout man of the cloth, Bayes was a pioneering statistician. His principal contribution to field is an approach to calculating the probability of uncertain events that begins with the assumption that all events are random, then refines the calculation as more information becomes available.

Bayesian inference, as this is known, is the basis of Autonomy's IDOL search platform. The software compares search queries and documents based on the statistical likelihood that they relate to the same topic.

The Autonomy story, of couse, is still unfolding. After acquiring the company for $11.1 billion in 2011, Hewlett-Packard accused Lynch and his fellow executives of fraudulently misleading it about Autonomy's value. The case is being investigated in the US and the UK, with Lynch strenuously denying all allegations against him.

See also: Webinar: The Near-Realities of AI-Powered Cyber-Attacks – available to download and view now

In the mean time, he has been looking for more commercial applications for Bayes's work.

Last year, he extended his personal investment in Featurespace, a company co-founded by Professor William FitzGerald, Lynch's former PhD supervisor at Cambridge University, which applies Bayesian inference to risk management.

Also last year, Lynch set up an investment firm called Invoke Capital, which counts many former Autonomy executives among its directors. In September 2013, the firm made its first investment – and once again Bayes looms large.

Bayesian cyber security

Darktrace, in which invoke reportedly invested between £10 million and £20 million, applies probabilistic statistical reasoning to cyber security. It was co-founded by Steve Huxter, a former security advisor to the UK government who had studied mathematics as an undergraduate (Oxford, as it happens).

"I'd known about some of the clever ideas coming out of Cambridge University, looking at different applications of recursive Bayesian estimation," he says.

The idea behind DarkTrace is to use Bayesian inference to model "normal" behaviour on a company's corporate network. Once that model has been built, any unusual or suspect behaviour can be identified in "real time".

This, says Huxter, is the only viable approach in the current era of information security, in which criminal and state-backed hackers will sit undetected in a company’s network searching for valuable data.

"I've seen security threats grow in sophistication dramatically, even in the last three years," he says. "But the products that are supposed to help mitigate these threats have stayed in neutral."

The perimeter approach to security – "locks and walls", as Huxter describes it – is broken, he says. And security log analysis is retrospective, he argues: if an attackers has compromised your network, analysing the logs is too little, too late.

Other so-called "behavioural" mechanisms, which attempt to identify suspicious activity, are rules-based, and therefore cumbersome to manage, Huxter says.

Darktrace sells an appliance that plugs into a company's core network. It conducts packet-level analysis to build up a picture of normal activity, and alerts security staff as soon as it spots any deviation from that activity.

The analysis can be concentrated on particular departments that may be at greater risk, Huxter explains. “If you are a manufacturing company, and a lot of your key assets are in R&D and intellectual property, we might focus on that areas more than others,” he says.

The technology can be used actively to entrap attackers who may have breached the corporate network.

“You can do things like create completely new components of a network, honeypots if you like, and put some fake, high-value data in there,” he says. “If you see that data moving around the network, it's a pretty strong indicator that something dodgy is happening.”

See also: See also: Cyber security – do you know where you stand?

Darktrace is working with a number of FTSE100 companies to implement the technology, Huxter says, and plans to use its new investment to expand into the US and mainland Europe.

The perennial complaint about the UK technology industry is that it fails to convert is world-beating academic research into viable companies. Lynch and his colleagues could disprove that, by building not just one successful company on but a cottage industry on the back some quite rarefied mathematical research.

Here’s hoping, for the UK technology sector’s sake, that nothing happens to undermine his authority before the likes of Featurespace and Darktrace get their chance to shine.

Pete Swabey

Pete Swabey

Pete was Editor of Information Age and head of technology research for Vitesse Media plc from 2005 to 2013, before moving on to be Senior Editor and then Editorial Director at The Economist Intelligence...