Hadoop will be the operating system for the data centre, says WANdisco

In 2000, Dr Yeturu Aahlad quit his job as head of distributed computing at Sun Microsystems to work on a thorny computer science problem he wanted to solve.

The problem was that active-active replication, in which two or more data sets are kept in perfect sync in real-time, did not work over a wide-area networks (WAN).

Aahlad solved that problem by applying the Paxos algorithm, a technique devised by computer scientist Leslie Lamport in the 1980s as a method of establishing consensus between distributed IT systems.

In 2005, Aahlad met UK IT entrepreneur David Richards, at a party in Silicon Valley, and the pair decided to put Aahlad’s breakthrough into practice. They set up WANdisco (short for WAN distributed computing), and named the company’s core technology the ‘Distributed Co-ordination Engine’, or DConE.

DConE allows multiple instances of the same application to run on entirely separate infrastructure, with data replicated across all instances in real time.

The first commercial opportunity WANdisco went after was distributed software development.

A common issue in geographically disperse team of programmers is maintain a single “master” code base while developers around the work on component parts, especially in the face of server failures and network outages.

By implementing code-sharing applications such as CVS and Subversion on DConE, businesses can make development more effective and more resilient. The code base is replicated across a cluster of servers, so that if one server fails no code is lost and developers can keep working.

The technology won customers including HP, Intel and Lockheed Martin. In 2009, the company needed to expand – and decided to base its new headquarters in Sheffield.

“It's probably the best business decision I've ever made,” says Yorkshire native Richards. “The cost of living isn't as expensive as London or San Francisco, and there’s a significant talent pool in the North of England.”

Investing in Hadoop

Fast forward to June 2012, and the company launched on the Alternative Investment Market (AIM) of London’s stock exchange. It raised £15 million in the IPO, which was reportedly four times oversubscribed.

In November, WANdisco used $5 million of that capital to buy its way into another field of distributed computing – Hadoop. It acquired AltoStor, a Silicon Valley big data storage start-up whose founders, Dr. Konstantin Shvachko and Jagane Sunda, were two of Hadoop’s original authors while at Yahoo!

Hadoop is mostly discussed in the context of “big data” analytics projects, but Richards says the distributed file system that underpins Hadoop, HDFS, has far reaching applications across the enterprise.

"Hadoop is going to be the operating system for the data centre," he says, "Arguably, that's Linux today, but Hadoop is going to behave, look and feel more like an OS, and it's going to be the de-facto operating system for data centres running cloud applications."

Combining AltoStor’s Hadoop know-how with DConE has allowed WANdisco to address a critical weakpoint in the big data framework, Richards claims.

NameNode is a component of Hadoop that keeps track of where each item of data is stored. It is a single point of failure for Hadoop systems, Richards says, as without it, the distributed file system falls apart.

In February this year, WANdisco announced Non-stop NameNode, a product that it claims renders Hadoop systems impervious to NameNode outages. “Even in the event of an electrical storm flood or terrorist attack, it means data is instantaneously available anywhere else,” Richards says.

Today, WANdisco announced that it has won its first Non-stop NameNode customer, an unnamed tier 1 telecommunications provider in the UK.

It also reported sales for 2012 of $6 million, up 56% from 2011. WANdisco made a $3 million loss for the year, although that included $2.5 million in costs associated with the floatation.

Evidently, the market is confident that WANdisco can make a dent in the potentially massive big data market, as the company’s share price rose 3% following the announcement.

Ed Reeves

Ed Reeves co-founded Moneypenny with his sister Rachel Clacher in 2000. The company handles more than 9 million calls a year for 7,000 UK businesses and employs almost 400 members of staff. Reeves remains...

Related Topics

Big Data