From data to insight

Businesses and governments have always hoarded information. The advent of computing in the 1950s greatly improved their ability to store information, to move it around and to analyse it. And as computing progressed, those organisations that knew how to use it became better at what they do as a result.

Five decades down the line, and the amount of data that most organisations have accumulated is nothing short of oceanic.

But the availability of useful information – information that helps employees at all levels to make better decisions and to do their jobs more effectively – has failed to keep pace with the data explosion.

Improving the ratio of data to useful information was a running theme throughout Information Age’s recent Information, Communication and Collaboration event, part of its Autumn Forum conference.

Traditional technological approaches to that challenge were discussed at length, most notably the use of business intelligence and analytics (see page 28). But an emerging theme was the way in which a human organisation converts data into useful information by communicating and collaborating around that data.

Collaborative conversion

The keynote address, from Jeremy Ruston, head of open source innovation at BT, challenged delegates to think about the influence that software systems have over employee behaviour and work patterns.
Ruston observed that, while stockpiling information in vast repositories (broadly speaking the main job of IT departments since their inception) may preserve an important record of information, it doesn’t necessarily make that information useful.

“Many of the systems that companies build feel like a massive warehouse in which you can’t see the walls,” he explained. “Its hard to know where the reliable, timely and relevant information is to be found.”

At their best, Ruston said, collaboration systems such as wikis allow users to help one another navigate information. In terms of deriving value from data, he argued, the job of annotating, tagging and flagging up useful information to colleagues is just as important as creating and storing that information in the first place.

However, there are considerable challenges that prevent this kind of collaboration from taking flight, Ruston explained. The first is participation: countless are the collaboration software deployments initiated in good faith that end up unused and unpopular.

Participation is often discouraged by a highly protective attitude towards data, Ruston said. So concerned are they for the sanctity of the central record of data, that many organisations will put measures in place that prevent people from changing it. “It turns out that this approach often condemns your data to becoming irrelevant, because if people can’t update data it will fall out of date,” he explained.

Another threat to collaboration within large organisations is the reliance on communication patterns that are inappropriate for significant numbers of people.

“We often think that collaboration is the purposeful, polite consultation with other people,” Ruston said. “In our everyday lives, we seek people out, have a conversation with them, and as a result of that take action; it’s a fundamental pattern in the way we interact and it works quite well, especially in small organisations. The problem is that it doesn’t scale.”

Employees of very small organisations tend to communicate en masse but as a company grows, people become selective in who they communicate with. This is the process by which divisional fractures begin to divide a company, and departments begin to operate at cross-purposes. “Once you’ve driven a wedge between departments, it’s very difficult to reunite them,” Ruston said.

In large organisations, Ruston argued, there is great value in giving employees the tools to publish information about what they do, and allowing their colleagues to either subscribe to that information or not. He gave the example of a global business that publishes the travel plans of its senior employees. If an executive is planning a trip to a certain city, local staff can schedule meetings with them during that trip, reducing the number of trips that need to be made.

The publishing paradigm better suits the nature of communication within large groups of people than the conversation paradigm, Ruston argued. Therefore, collaboration tools based on, for example, the RSS syndication standard can effect a fundamental shift in the way an organisation communicates internally – and therefore derives value from its information – at a relatively low cost.

Social selling

While communications software can be deployed in such a way that it encourages more effective patterns of collaboration, software can also be used to determine existing patterns, as the event’s second keynote, from Trampoline Systems CEO Charles Armstrong, expressed.

Armstrong argued that a critical shortcoming of today’s customer relationship management (CRM) systems is the inability to properly identify the key personal links upon which customer relationships rely.

However, appraising interpersonal relationships is now within the capabilities of analytical software, Armstrong explained. At its simplest, it can be done by analysing patterns of email exchange. For example, two people who exchange roughly the same number of emails can be said to have a closer relationship than two between whom the direction of email traffic is one sided.

This analysis can help to remove the risk from customer relationships, Armstrong explained. “If a customer relationship is hugely dependent on one person in the customer organisation to the extent that if they left then your ability to influence that customer would be significantly impaired, that can be identified statistically,” he said. “Similarly, if there are several key contacts within a customer that only one of your employees has links to, that can
also be identified.”

“These are potential hazards in a sales process that can be automatically removed,” Armstrong explained. Social analytics, he believes, represents the next major shift in the evolution of CRM applications.

Efficient communications

Armstrong argues that applying the computational power of analysis to communications can deliver great insight into organisational dynamics. Similarly, improving the flexibility of communications can introduce agility and speed into the way an organisation collaborates and shares information.

Paul Livesey, business development manager at communications consultancy Damovo, says that his organisation is as sophisticated an adopter of unified communications as exists today. Not only has it adopted the technologies associated with UC, but it has also undergone the cultural changes that the technology allows, such as remote working and real-time collaboration. That, he argues, has radically improved its efficiency as a business.

Even as Livesey was at the Information, Communication and Collaboration event, his team was working on a significant proposal for a new client. While waiting to take to the stage, Livesey conducted an instant message chat coordinating his team, and later conducted a conference call on his mobile to give approval to the sales documents.

“If it hadn’t been for unified communications technology, I wouldn’t have been able to leave the office today, because even if I could get connectivity I wouldn’t have been able to get through the firewall, and I wouldn’t have been able to see that my team are doing what they are supposed to be doing.

“In fact, without UC, I would have had to have been in the office every day for the past three weeks,” he added. “I’ve been in three times. That is what we mean by efficiency.”

While the paradigm of building large repositories of data was still very much in attendance, the above presentations gave delegates a glimpse of a new vision of information management, in which the flow and movement of information around the organisation is what converts it from mere data to useful insight.

Information in the cloud – the legal consequences

From a technological standpoint, it is now almost irrelevant where in the world a business stores its data. Records could be kept in a server room within the office or in a data centre thousands of miles away, and the difference in latency speeds would be immaterial to most businesses.

But as Bridget Treacy, a partner at law firm Hunton & Williams, explained to the Information, Communication & Collaboration conference, changes in the law often lag behind the development of technology. And at the moment, the incompatibility of regional data protection laws makes cloud computing a thorny legal issue.

At times, it can seem as though the law is being unhelpful, or standing in the way of progress. But as Treacy explained, it is useful to know the history of data protection laws in order to understand their rigidity.

“European data protection laws were developed in the aftermath of the Second World War, as a reaction to the secret reporting on individuals by the state that went on,” Treacy explained. “If you were unlucky enough to have a file kept on you by the state, you wouldn’t have had any right to check whether the data was correct or to challenge it if it was wrong. So these laws were very much made to protect the citizen from the state, and the right to data protection is held very dearly in Europe.”

Currently worrying European data protection regulators is the passage of data between legal jurisdictions, she explained. “Data could be stored in multiple different countries, and it could be subject to a number of different obscure, local provisions that allow agencies access to data processed on servers within that jurisdiction.”

Another cause for concern is the definition of the roles of ‘data controller’, who has legal obligations to ensure data protection rights are upheld, and ‘data processor’, who does not. “The line between the two roles is very blurred,” Treacy said. “Its important to monitor the relationship between parties as time goes on, because it can change.”

“There are legal issues that need to be addressed by both the vendor and the customers,” she said. “I would encourage both parties to think about these issues in a collaborative fashion. Frankly, everybody ends up with a much better solution – a solution that stands the test of time – if it is worked on collaboratively.”

Productive partnership

The field of software development has seen a revolution in the way people work together, both within and across organisations, namely Agile development, in which developers are encouraged to produce functionality enhancements quickly and frequently. Although closely linked to software programming, Agile is in essence an organisational innovation.

In order to build a new website for fans of its soap operas,, broadcaster Five turned to Agile development agency Unboxed Consulting. Speaking at the Information, Communication and Collaboration event, Unboxed’s CEO, Richard Stobart, explained how the company agreed to deliver new points of functionality every two weeks until the website build – based on a predetermined design – was complete.

The seven-strong development team followed a rigid daily process of identifying ‘story points’, (landmarks on the way to certain functionality), obstacles to the attainment of those story points, and approaches to overcoming those obstacles.

New code was tested as it was developed, and the functionality that was delivered every two weeks was ready for production. “There isn’t an extra stage of integrating new functionality into the application, or an extra testing phase,” explained Five’s technology manager, François Chabat. “Every two weeks new features can go live.”

At the end of every two weeks, the team assessed what went well and what didn’t, in order to continually improve the production process. In total, the entire project took 12 weeks, or six two-week iterations.

Unboxed charged Five a fixed price for time and materials, plus a rate for each ‘story point’ that was achieved and approved. Chabat said that, while the traditional model of billing – in which the client agrees a fixed price for the entire project – is more reassuring, “more often than not, it is not delivered on time because there is unknown complexity, and the price goes up”.

At first, the billing model that Unboxed proposed felt a bit risky, says Chabat, “because we are not sure what we are going to get”. However, it also meant that if Unboxed delivered only 60% of what it had promised, Five only had to pay 60%, so the supplier is incentivised to keep its promises. “It is not ideal,” said Chabat, “but it is infinitely preferable to the alternative. We believe the risk is managed much better [with this approach].”

Pete Swabey

Pete Swabey

Pete was Editor of Information Age and head of technology research for Vitesse Media plc from 2005 to 2013, before moving on to be Senior Editor and then Editorial Director at The Economist Intelligence...

Related Topics