Information Age: News, analysis & insight for IT & business leaders

If there is one Hollywood film that has intrigued IT executives over the past decade, it is Steven Spielberg’s deeply flawed, but technologically prescient, Minority Report.

The 2002 film, set in the mid-21st century, is a chilling advertisement for many of today’s nascent technologies: millions of remote cameras biometrically scan people as they go about their business; radio chips track everyone’s movements; personalised interactive advertisements cry out people’s names in synthesised voices across the shopping malls; and miniature robots conduct searches and scans in the service of the police.

But the film also has a daft premise that, while vital to the plot, undermines its credibility. A group of drugged, dehumanised individuals called “pre-cogs” are able to predict crimes just before they are about to happen. Using the sketchy information they provide, supercomputers whirl into action, calculate the location of the imminent event, and a Swat team is dispatched to arrest the would-be perpetrators moments before the crime occurs.

All of this raises moral and philosophical questions, some of which the film addresses in a Hollywood ,apple-pie kind of way. But to those of us fascinated by the application of IT, there is another interesting question it raises: Without the dramatic device of the “pre-cogs” to predict the future, would it still be possible to do the job using networked supercomputers, linked to billions of real-time information sources and aided by advanced, real-time analytics and event-processing tools?

It is unlikely, perhaps, that technology will ever be developed to report on criminal intent in real time. But what about buying behaviour? What about the actions of stock market traders? Of governments or armies in times of international crisis? Can IT actually give us the power to predict?

Depending on the degree of prediction, many would say this is not exactly blue sky. Setting aside time travel and quasi-religious pre-ordainment, the only practical way in which events can be predicted is to sit down and try to work it all out – some might call it “analysis”. The simple idea is that everything that happens is the causal result of a preceding event, or a complex pattern of events. In this way, someone with enough knowledge and understanding of all the relevant causes that trigger an event can, in principle at least, predict it. People have been trying it, with varying degrees of success, for several millennia.

But this is where it gets interesting: Up until now, the analysis of human behaviour is more often than not conducted retrospectively, and it takes time. And while some people are skilled in predicting certain outcomes in certain discrete situations, that doesn’t scale to universal applications. But that may – just may – be about to change.

In his new book, The Power to Predict, Vivek Ranadive, the CEO of Tibco, the enterprise integration software supplier, argues that the world is about to enter a new phase in which governments and businesses are able to routinely and systematically collect all the relevant data relating to an event, as it is happening, and then predict what will happen next. They will be able to do this using a combination of real-time, event-driven systems and complex event processing (CEP), a technology for analysing outcomes from complex patterns of events (see David Luckham’s book, The Power of Events, reviewed in 2005 in Information Age).

Examples of this in action today are thin on the ground, but Ranadivé does like to cite the example of Harrah’s Casinos, which is run by Ranadivé’s old college mate Gary Loveman and operates gambling venues including the iconic Caesar’s Palace in Las Vegas. Harrah’s stores analytical data about customers’ behaviour, and attempts to maintain its customers’ positive mood by serving up special offers and support in response to events – such as a heavy gambling loss.

At the launch of his book in Paris recently, Ranadivé encountered sceptics: one pointed out that attempts to predict stock market behaviour have always failed on a grand scale. Others said mistakes would be made as data sets could never be complete.

Ranadivé counters that he is not trying to predict the weather – he is merely saying that the world’s IT systems are about to move ahead of real time, and, moreover, they have the technology to do it.

But here is a prediction: if the first systems produce results – and at Harrah’s apparently they do – then the models will get bigger, the number of events subjected to analysis will multiply exponentially, and some very big and scary predictions will start to look eerily accurate.