Information Age: News, analysis & insight for IT & business leaders

Spotfire's alternative approach was an early example of a so-called 'in-memory' business intelligence tool, wherein data to be analysed is moved into random access memory for greater speed of analysis. 

It is not, however, an entirely in-memory system, examples of which are prone to crash when applied to high quantities of data, Hopper claims. Instead, a summary aggregation of data is brought from disk into memory and presented visually so that it can be filtered and explored. When the user ‘drills down’ into a more detailed subset of the dataset via the visual interface, that underlying data is pulled from the disk into memory without, Hopper insists, any disruption to the user’s all-important train of thought. 

Most conventional business intelligence tools have ‘drill down’ functionality, but Hopper argues that these are still simply reports at lower levels of granularity; the user is not using the visual interface to navigate the actual data.

“Traditional business intelligence systems provide to the report the absolute minimum amount of information it needs to render the chart,” he says. “The ability to span multiple in memory instances that populate themselves in response to visual cues is something that is very highly differentiated [for Spotfire].”

Hopper echoes Mackinlay's point that traditional BI systems curtail visual exploration by requiring that the user knows what view will be most enlightening before generating a given chart. “The key difference between a visual analytic tool and a more traditional business intelligence tool is that with a BI tool you need to know which questions you want to answer in advance,” he says. “And you might be able to create the most glorious report to answer that specific question, but nine times out of ten somebody is going to look at that report and have another question and that is going to require that you go through the whole cycle again.”

Superficial understanding

Few says that although the field of data visualisation has been given a bad name  “because some of the stuff sold under that term is just horrid”, suppliers such as Tableau Software and TIBCO Spotfire are gaining recognition, and users are beginning to realise what can be achieved.

“When I show people some of the more enlightening techniques that can be used in analysing data, there really are ‘a ha’ moments when they realise that their reach could be dramatically extended if they just had a better tool to work with,” he says.

This has not escaped the notice of the conventional BI suppliers, says Few, and some have launched what they claim to be ‘visual analytics’ solutions. “The larger players are running into sales situations where their customers are saying ‘why can't you do that’, and so they have been making efforts to emulate what can be done in tools like Tableau and Spotfire,” he says. “But because their understanding is superficial their efforts are just disastrous.”

Given the above mentioned companies’ academic backgrounds, one wonders why the conventional BI suppliers don’t simply hire the brightest and best brains in data visualisation to make their products more accessible and more exploratory.  “They could certainly go out and hire the right people, but they haven't done that yet and I don't know why,” remarks Few. “Maybe in some cases, their egos are too big to accept the fact that they don't have the skill set.”