#
 
ANALYSISDISASTER RECOVERY

Lessons from 7/7

The July bombings tested business continuity plans to their limits. The British Medical Association, the Metropolitan Police and Marks & Spencer describe how their IT systems coped when terrorists attacked London last summer.

The BMA on 7 July 2005

Business continuity plans are a waste of time and money; they are pointless. This, says Poli Avramidis, the director of information management and IT at the British Medical Association (BMA), talking at the recent Business Continuity Expo, was the prevailing view at his organisation when he joined four years ago. “We had nothing in place, but external auditors said if we were going to do risk governance we needed a business continuity plan [BCP], so we created one.”

That plan, along with that of many other organisations, was put to the ultimate test on 7 July 2005 when four bombs on London’s transport system killed 52 people and brought the capital temporarily to a halt. Mobile telephone networks were overwhelmed in the first hours by high call volumes and the O2 network around Aldgate Tube, where one bomb went off, was diverted for the emergency services’ use.

The BMA was dragged into the chaos when a bus exploded outside its offices and its doctors rushed to treat the wounded, and the Metropolitan Police Service was compelled to keep its business running despite failing technology and traumatised staff. But many organisations could do little more than send their staff home.

Retailer Marks & Spencer became embroiled in the crisis when it opened the doors of its Edgware Road branch to commuters coming out of the tube station. Trevor Partridge, the head of business continuity at Marks & Spencer describes the absurd scenario of M&S’s director of operations being forced to queue for a phone box to help direct the retailer’s response.

“Mobiles, BlackBerrys, Bluetooth headsets: they are all great pieces of kit but they didn’t work that day, they were useless,” he notes.

The Met’s network, however, had enough capacity to enable it to continue when other organisations could not. It is also in the process of moving away from its legacy MetRadio radio system to the nationally-utilised Airwave digital radio network. But on 7 July, it still had to rely on its close links with British Transport Police to enable it to have electronic communications underground, since neither MetRadio nor, in the future Airwave, facilitate this.
 "The phones groaned. We took 43,000 calls in an hour."

Commander Mick Messenger, Metropolitan Police

As with the mobile networks, high call volumes presented problems for the police. The Police Casualty Bureau – the Met’s team that deals with the public, hospitals, victims and their families during major incidents – was inundated with some 43,000 calls in the first hour it was open.

The Met had learnt some lessons from when the Bureau was overwhelmed by thousands of people calling in after the tsunami that hit south-east Asia on Boxing Day 2004. But on 7 July, the phone systems still “groaned a bit”, says Commander Mick Messenger. “We thought we’d be able to cope, but it’s a problem not easily solved because you have to retain the balance between the high volume of calls and the need to treat each individual as a core part of the investigation.”

Best laid plans

Even with the best-laid contingency plans, many organisations’ technology struggled to meet this unprecedented challenge. Despite the complacency about business continuity Avramidis found when he had arrived at the BMA, within a year his team had constructed a plan aimed at keeping in production its weekly magazine, the British Medical Journal (BMJ), as well as the BMA’s everyday operations.

While the explosion on the number 30 bus outside its Tavistock Square offices initially caused panic and fear – Avramidis paints a stomach-turning picture of windows being blown in and body parts landing on desks – within around 30 minutes its continuity plan was invoked.

The BMJ has 25 seats reserved at a Croydon location for instant recovery, but it did not quite work out as intended. “There was no transportation out of London to get staff there, and even when we could get people in, the systems didn’t work,” Avramidis explains. “Instead of being an instant recovery it became a 13-hour marathon.” Without a usable remote site, “the BCP was just a piece of paper”.

When staff finally reached the Croydon office for the BMJ they found the instant kick-start had failed. “I had to send some of my staff over to get it to work and they finally managed it by about 3am. It was a bit of a waste of space,” bemoans Avramidis.

The BMA’s headquarters was a crime scene, with access strictly limited by the police, meaning problems with back-up systems could not be easily fixed because of time constraints. But in the end, the BMJ was published to deadline. The BMA itself survived as well as it could have hoped. It was able to transfer operations to its Glasgow call centre for the subsequent 12 days it was shut.

Even outside the centre of the attacks, tried and tested systems failed. At Marks & Spencer, a crisis management team had been established for such a scenario, but initially it was unable to convene because the members were in different locations – central London and Heathrow – and the audio-conferencing system did not work.

“We had tested it and it had been fine,” explains Partridge, “but it didn’t work when we needed it.” Now, Marks & Spencer has outsourced the audio conferencing to a managed services provider to make it more robust; it can now link up to 150 people together at any given time.

Furthermore it has established a 4,000-square-foot crisis management centre at its data centre at Heathrow, so the team can assemble there if central London is unsafe or unviable.

Continuity counsel

Business continuity plans need to be focused on people, not just assets:

• Use the latest wireless tracking solutions to provide real-time
location and status of mobile assets

• Set up several paths of emergency communication with mobile transport crews, such as email, voicemail, cell phones, pagers and PDAs

• Strengthen back-up operations control and dispatch centres that can be brought online instantly

• Balance investments and the management attention dedicated to systems that protect passengers, transportation workers, mobile asserts, cargo and infrastructure.

 

Source: Gartner Research 

Lessons learnt

Such strains have led these organisations, and doubtless many more, to review and improve their business continuity plans.

The BMA has now refined the process of its evacuation, adding things like ‘battle boxes’, which hold critical contact information, and also radio communications after the mobile and BlackBerry networks were crippled. M&S has developed a portal on its intranet so its staff can be kept up to date both at home and at work about what to do in a similar situation.

The Met has now virtualised its Police Casualty Bureau phone system so that staff in other areas of the country can deal with the calls, enabling the local officers to deal with other, more pressing, commitments.

What became apparent during 7 July was the need for organisations to consider their employees’ emotional well-being when trying to keep their business running.

The BMA offered counselling to all its staff and lost just one out of 330 to long-term sickness. Transport for London worked hard to convince its staff they would be safe working on the tube network after the carnage they had just experienced.

“Madrid’s transport system took six months to get back to anywhere near normality after the March 2004 attacks, and over a year to regain its pre-March revenue levels,” explains Geoff Dunmore, London Underground’s operational security manager. “We were back to normal by September and exceeding our previous revenue levels.”

The Met’s Commander Messenger says private companies also did a good job at informing and protecting staff: “The internal communications in the City were brilliant,” he says.

Although back-up plans did not always work as expected, the BMA, London Underground and the police all agreed that London proved particularly resilient: the FTSE100 stock market bounced back within a day, 85% of the transport services were able to run straightaway, and the city, despite its shock, went back to work.

Further reading

By Tim Bradshaw,