It is increasingly common to hear holidaymakers bemoan the time they’ve wasted by standing in security queues at airports. Mostly it is low-level grumbling, as travellers recognise the need for vigilance. Nevertheless, for the 17,000 passengers left stranded at Los Angeles’ LAX airport on the first weekend in August, 2007, complaints were loud and vociferous – such are the consequences when it is an IT security system that goes for a Burton.
At first it appeared that the complete IT system crash that stranded the unfortunate travellers might have been terrorism – perhaps the most damaging act of ‘hacktivisim’ yet recorded.
The truth turned out to be a little more prosaic. A single network interface card (NIC) on one desktop on the U.S Customs and Border Patrol’s network had malfunctioned. This caused a domino effect, with subsequent network access cards becoming overloaded, which resulted in complete network failure.
It took nine hours to repair the system, only for it to fail once more the following day.
It emerged later that although all data passing over the network was backed-up to data-centres in Washington, the connection to the back-up databases also relied on the local area network. When that LAN became overloaded, back-up stopped, hence the nine hour repair job.
But how a single card took down the entire local area network remains something of a mystery. “Usually when a card like that malfunctions it shuts down,” said Ken Ritchhart, of the CBP’s IT department. “In this case, it went crazy and brought down the network.”