True grid
- Reduce text size Decrease text size
- Increase text size Increase text size
- Print article Print
- Jump to comments Comment
- Share this article Share
- Email article to a friend Email
Today, tomorrow, or a decade away: when will grid and utility computing be ready for enterprise IT?
| |||||||
The pattern could not be clearer. Businesses are buying IT in smaller increments; projects are staged and phased, split into chunks, each based on proven technology and each needing to show demonstrable return on investment before the next phase gets the go ahead.
So why, when organisations are thinking in terms of cautious steps, are the major industry vendors asking them to think big and long-term? Really big and really distant.
Grid computing, utility computing, organic IT: the models may differ in substance, but the path to those ends (see box, 'Evolving vocabulary') requires a complete re-architecting of the IT infrastructure and a five to 10 year overhaul.
"The simplest way to characterise it is this: we have viewed computers as things we attach to networks; now computers will be built from networks. And if an IP network is your backplane, then the whole Internet can become your computer," says Greg Papadopoulos, CTO at Sun Microsystems.
But why, in the current climate, would the largest systems vendors be talking revolution? The fact is they are doing so for the same reasons that business has become
| ||
Systems are, to a frustrating degree, under-exploited. Average processor utilisation levels stand at 15% to 20%, and when peak periods do occur further headroom has to be obtained by adding to the 'server farm'. As a result, administration - and software costs - have become painfully high. At the same time, that structure means applications are tied to a set environment, unable to access additional processing capacity when the need arises. And any failure in these silos means the resident application is out of action. In the words of Forrester Research analyst Galen Schreck: "Today's server infrastructures are a mess."
And senior management are hardly blind to the fact. "There are lots of very large corporations out there which are quite annoyed with hardware providers because they were oversold to during the boom years. And they know that with the distributed server estates that they have today, a lot of those servers are 30% utilised," says Una Du Noyer, head of infrastructure services at IT services company Cap Gemini Ernst &Young (CGEY). "So what we are seeing is a market push back onto the hardware provider to fix the problem, with the message: 'We are not buying any more of this stuff until it is able to be utilised more efficiently'."
Frantic rush
The thinking behind the major vendors' response to that - their different attempts to come up with a way in which IT can take a giant leap forward towards becoming a flexible, on-demand resource - may be ambitious but is not totally new. The notion of grid computing (what IBM's general manager for grid, Tom Hawk, describes as "the lashing together and sharing of under-utilised resources") has been explored in scientific and technical computing circles for decades, even though the actual ability to construct grids is perhaps only five years old.
What is more, it is only in the past year that companies such as IBM, Hewlett-Packard (HP) and Sun have started to pursue the enterprise grid computing agenda in earnest - whether based on a grid distributed across the Internet or a set of internal resources viewed as a virtual pool and provisioned to meet changing needs.
And not just the major systems vendors. Some of the loudest voices in software and services - companies such as Oracle, Computer Associates (CA) and CGEY - accept that some kind of utility or grid computing architecture is the future. But claims of corporate penetration by grid and utility computing beg a deeper analysis, a reality check. Just how universal can the new models be applied in a commercial environment; are there evolutionary steps to achieving the promised end result; what lessons have been learned from the application of such techniques in academic and research contexts; and to what extent is it credible that vendors will work together to create an infrastructure that is effectively vendor neutral?
Poll positions
Certainly, the tag of 'vaporware' that has been applied to grid and utility computing is no longer wholly justified. Over the past few months, vendors have been at pains to identify businesses that have implemented several key parts of their new architectures.
Grid is already being applied to business problems that involve the processing of transactions or data in a highly parallelised way. "The dead ringers for enterprise grids are problems that require highly parallel processing," says Hawk. And certain resource hungry applications - in life sciences, financial services and product design - lend themselves particularly to that.
Aside from applying raw power to compute-intensive applications, grid has also helped organisations defer capital expenditure by making more out of existing systems.
"If an organisation has increased computing requirements, and they know a lot of their servers are under-utilised, by 'gridding' them together they can actually increase utilisation from 30% to up to 80%, which means they don't have to go out and buy new boxes," says CGEY's Du Noyer.
At Abbey National, a deferral of capital spending was very much a factor in its initial grid implementation. The bank's derivative-based products require vast amounts of data processing. "We have to process very complex models, which can involve as many as 50,000 calculations per trade," says Noel O'Mahony, head of IT Financial Products. "We needed to find a more effective method than our existing approach. We have some very powerful machines that were only in use around 10% of the time, and we wanted to tap into that idle processing power and to distribute the processing over a cluster of 50 servers [using grid software from DataSynapse]."
Similar drivers influenced the grid adoption at investment bank Morgan Stanley, which is migrating certain analytical applications from its mainframes to a grid of several hundred Intel servers.
Outside of financial services, pharmaceutical giant Novartis plans to make use of the surplus processing power of its researchers' PCs to speed drug discovery. The Swiss company is using United Devices' grid software to tie together and exploit unused processing power on the 2,700 machines. The cost in terms of grid software for a structure that will deliver the equivalent power of a supercomputer: just $400,000.
What these examples show is that the early uses of corporate grids is limited almost entirely to applications that are similar in form to those already being run in the scientific and technical computing arena.
But grid researchers believe that most business applications could use grid. "The bulk of business computing involves transaction processing, and although a group of transactions may be linked, it is unusual for a complete processing chain to be linked," says Mark Parsons, head of GridStart, a European Commission-funded grid coordination programme at Edinburgh University. "For example, when an airline reservation system is booking a flight, the series of transactions it executes are typically processed all on the same machine. But a thousand of these bookings might be being processed at the same time and you could easily distribute those discrete groups of transactions around the grid." Nonetheless, even the major vendors concede that grid is not necessarily ready for that wider role today. "In a commercial setting grid is not the model of work distribution that meets most people's needs at the moment," says Sun's Papadopoulos.
Held back
Such thinking is reflected in a recent survey by Forrester. It showed that while management at the vast majority of large IT sites had heard of the new computing models, only a quarter had any serious thoughts of adopting them soon.
One reason for that is lack of clarity. "All the vendors have been successful at establishing a level of awareness of their message," says CGEY's Du Noyer. "But there is a lot of confusion around what is meant by grid, by utility,
| ||
Some caution also stems from predictions of how long it will take for all the necessary pieces to be in place. Even some of its most ardent proponents suggest the new model of IT is some way off. "The goal is to provide ubiquity and easy access [to grids]. We are, of course, many years away from that vision," says Carly Fiorina, CEO of HP. She reckons it will be three to five years before companies can take full advantage of commercial enterprise-class grids.
Others see it as an even longer journey. "The ideal world is at least 10 years away," says Yogesh Gupta, CTO at CA. "The credibility of the tech industry is not in good shape," he adds, and that situation makes it difficult to sell any radical new approach, no matter what its long-term potential.
Challenges ahead
Even if that caution is overcome, there are a whole set of technical obstacles on the road to achieving the ultimate vision of a true grid and the ability to provide IT as a utility service.
One is the complexity and immaturity of grid software.
To date, the main software platform for developing grids has been the open source Globus Toolkit produced by an alliance of some of the world's top universities. The first two versions were almost exclusively about creating grids that supported scientific and technical computing. However, Globus Toolkit Version 3, released in July 2003, is much more aligned with commercial computing and its increasingly preferred application deployment and delivery mechanism - web services. Most importantly, it implements the Open Grid Service Architecture, an interoperability framework developed by IBM with some help from Microsoft, to provide for the running of web services over a grid.
"Over the past 18 months ago, the focus for building grids has moved from a pure scientific base to a web services approach," says Parsons. "The grid services view and the web services view are actually smashing together at the moment. The difference between them is very, very small."
The relationship between web services and 'grid services' is critical. Says CGEY's Du Noyer: "With web services you have a nice thin client interface to applications. The applications can be provided from anywhere [on the network], because you have a separation of the presentation from the applications and the information behind it. If you take that another step further, one of the goals of grid is that you can run an application or components of an application anywhere, depending on where there is more resource available. That layered approach allows you to virtualise the provision of your services. You don't know where the application is running, you don't know where the data is held."
But the shift towards web services by Globus means a lot of the code is still immature. "This is a very big block of code, and the sheer size and complexity of it increases the risks," observes HP's Fiorina - the risk of failure, of error, of security breach. "And those risks need to be worked out before a CIO can get close to considering [using] it," she adds.
Support for heterogeneity is another vital issue that needs addressing, and most systems vendors (at least in principle) agree that it is a prerequisite for the wider roll out of grid. However, given the multiple number of operating systems, platforms, databases and applications, almost every implementation of grid to date in an enterprise setting has used a uniform set of platforms.
A desire to keep complexity as low as possible at this stage has also meant that organisations have built virtual pools of resources from internal systems. HP's Utility Data Centre (UDC) takes that approach. Using UDC, the organisation's servers and storage resources can be 'provisioned' to different applications at different times depending on variations in their workload. However, unlike pure grid, the applications stay whole - there is no hint of splitting up tasks and executing them across multiple, distributed processors. Among the couple of dozen early users of the UDC model are Philips' Semiconductor division and film production company 422.
As that suggests, many of the key software components for establishing, running and maintaining a limited utility environment are already being delivered. "We now have pieces that allow you to do the virtual re-plumbing of systems and connectivity, with policy-directed resource allocation," says Papadopoulos at Sun, which is currently fleshing out its N1 grid product line.
Products addressing the many other aspects of grid are just appearing. HP, for example, has released a grid topology designer for mapping out resource needs and submitting these to the grid for fulfilment. It has also released Smartfrog, a toolset for remotely configuring resources on the distributed computers that make up the grid.
"These things do take time and hard engineering," says Papadopoulos.
That is not just being done by the large systems vendors. While all have expanded their grid and utility portfolios with multiple acquisitions, they are also partnering with some of the specialists in this area, companies such as Akiva, Entropia, Platform Computing, DataSynapse and United Devices.
Many of the elements of the software stack that will have to be 'grid-enabled' are also just appearing. Trying to capture some of the early momentum behind the new architectures, Oracle launched 10g in October 2003, a version of its database management system that spreads processing across multiple nodes, thereby increasing performance, scalability and reliability.
CIO opportunity
As such activity indicates, the journey to grid is well underway. CGEY's Du Noyer describes grid as evolving along several different strands. The first, she says, involves organisations embarking on consolidated, pay-as-you-go, on-demand computing, resulting in a more utility-like environment. The second is the use of grid within the organisation, and later its ability to connect the organisation collaboratively to partners or a supply chain of research partners. The last stage is where organisations are using grid as a mechanism for true utility computing, where supply is pumped in from any source.
There is certainly a chance here to turn IT into the kind of service that most businesses would want. "There is a huge opportunity for the CIO to prove that IT assets can be managed in line with the priorities of the business," says Ian Curtis, marketing director for HP in the UK. "The infrastructure to date has made it difficult for the CIO to do that."
And the IT industry recognises that it has to go a lot further than in the past if much of its lost credibility is to be restored. "For the first time our energy is focused on inventing something far more radical and far more profound than a 'killer app' or a 'hot box'. Our energies are focused on making computing more useful, more competitive, and less expensive: On synchronising IT and business," says Fiorina.
Is there is a danger that the move to enterprise grid stalls and is restricted to niche, compute-intensive applications? "There is a chance, but I think it is small," says Hawk. The momentum behind it is too great; the potential gains in efficiency too alluring; and the advantages of having IT resources 'on tap' too beneficial to business, he says. So the question is not if, but which elements when.
