Deep below the surface of the Antarctic ice, a vast array of optical sensors is searching for a faint blue light.
This light could indicate that the ice has been struck by a neutrino, a sub-atomic particle that is notoriously difficult to detect, that may have travelled through space. By analysing the precise behaviour of this light, scientists from the University of Wisconsin hope to discover new insights into the make-up of the distant universe.
Data produced by the IceCube, as the frozen telescope is known, is collected and crunched in a 150-server data centre located not far from the South Pole, where temperatures can drop as low as -80 degrees Celsius.
Happily for Paul Wisniewski, systems engineer for the project, most of the administration can be performed from the university’s US campus, although he is a frequent visitor to the South Pole.
Surprisingly, given that the data centre is located in the coldest place on earth, one of the challenges Wisniewski must contend with is overheating servers. “Believe it or not, we’re faced with heating and cooling issues,” he says. “We can’t just bring in cold air from the outside, it has to be conditioned. The big thing is that there’s zero humidity out there.” Wisniewski’s solution to this has, by his own admission, been “makeshift”, and includes such measures as attaching humidifiers to systems in an attempt to moderate temperature.
Just like living on the icy continent, managing an IT operation in Antarctica is a matter of making the most of scarce resources. The IceCube’s network connection to the university is conducted via a NASA satellite network, and is only available for six hours each day as the satellites pass within range of the Antarctic.
During these hours, Wisniewski explains, 35Gb to 50Gb of mission critical data is transferred to the university’s data warehouse via a US Department of Defence site in New Mexico.
A second satellite network is permanently available for communication purposes, but this has a finite capacity and allows only “a limited amount of email” to be transmitted each day. Non-essential data is stored at the facility on magnetic tapes.
Despite the overall project’s $370 million budget, Wisniewski must also keep IT costs to a minimum. This explains the recent decision to move the servers from Red Hat’s commercial open source distribution of the Linux operating system to a free version called Scientific Linux
Wisniewski will have to travel to the South Pole for this switchover, a journey that can only be made during a certain time window. “The construction season is from the beginning of November until mid-February, and only during that time can the planes get in and out,” he explains.
Outside that time window, the IT support that Wisniewski can provide is limited to what he can do remotely. For this reason, the skeleton crew that mans the facility during the winter months is given extensive hardware training before they arrive. “By the time they get down there, they’re pretty well ingrained in how these things work,” Wisniewski explains.
The IceCube’s environmental conditions necessitate a low-touch approach to data centre management. Elements of this approach may in time become more commonplace, as data centre facilities become more remote and their management more automated.
