Before computing can truly become a utility service, computer vendors must build an infrastructure that can deliver services at the flick of a switch. That, say Sun Microsystems and Hewlett-Packard (HP), is what their new grid exchanges will do.

HP and Sun are in the vanguard of a movement that believes computer capacity is destined to become a commodity: an on-demand service, delivered over a network as simply and flexibly as today's power grids supply electricity. Ultimately, these utility theorists claim, building a private data centre to generate computing capacity will become about as economic and as popular, as building a private power station.

This idea of computing as a utility is not entirely new. During the 1960s, bureau services leased capacity on a time-share basis to organisations that had neither the need for, nor the resources to own and maintain, a mainframe of their own. But this market withered away as cheaper, distributed servers allowed companies to acquire their own capacity in more manageable and affordable chunks.

Today, acquiring capacity on a server-by-server basis is still the dominant computing model, but it is far from ideal. Although modern servers are relatively cheap to buy, they have an expensive habit of proliferating, making them costly to manage and almost impossible to optimise. It would, say the utility theorists, be cheaper and easier if companies only consumed the capacity they needed when they needed it, and at prices dictated by economies of scale that only a shared service utility can provide.

Grid computing utilises a variety of systems automation technologies to allow multiple computers to be treated as one large one. This 'virtual' computer resource can then be scaled up and down as required, and shared amongst different groups of users – even between different sites and organisations.

Today, Sun is already able to rent its Sun Grid infrastructure to customers for $1 per CPU per hour (see box). This is arguably the world's first published utility computing tariff, but it is only the beginning. Last month the company announced plans that, with input from electronic trading platform operator Archipelago Holdings, will allow customers to bid online for capacity as if they were buying surplus office space. Power and telecoms companies trade kilowatts and minutes in much the same way.

Sun has offered no launch date for this exchange service, and may yet be beaten to the punch: HP already operates its own pilot marketplace (see box); IBM claims to have an exchange service in the pipeline; and even Oracle is considering offering its ebusiness suite on a grid model.

More important than who is providing the service is who might use it. So far, interest in grids has largely been restricted to companies such as life-science researchers and digital animators with wildly fluctuating needs for occasionally massive computing resources. Such users are always likely to be net consumers of grid capacity.

To make the economics of public grids work properly, exchange operators must attract more mainstream customers who have significant computing 'capital' of their own, and are prepared to trade that 'liquidity' on an open market.

Cultural change

"That is the expectation we have," says Andy Ingram, marketing VP for Sun's scalable systems group. "Companies with excess cycles will put them into the exchange. The whole point is to match supply and demand globally – enabled by the global network. It is easy to set up the exchanges. The hardest thing is looking at some aspects of your computing environment as a utility."

The first of these cultural issues that need to be negotiated is the familiar and understandable reluctance with which many companies view the prospect of entrusting key resources to the hands of third parties.

"It's like the Internet," says Philippe Bricard, IBM's grid computing executive for EMEA. "Initially businesses didn't connect but after two or three years they did – not because of new technology but because the model matures."

Security has a key role to play here. Strong measures must be in place to prevent one grid user's data from being exposed to the processes of another and to prevent attacks from the Internet.

Sophisticated new systems management tools will also be needed to manage application licences, measure their usage, and allow them to be charged for accordingly.

However, Sun believes the key to grid acceptance lies in making it easier for companies to determine the price, and hence the value, of any grid investment. "Our view is that many suppliers in the technology industry have relied on mass inefficiencies and opacity to drive short-term profit," Sun president and COO Jonathan Schwartz wrote on his online diary the day the Sun Grid went live. "In the commodity world, it is all about price and transparency – that is at the heart of an efficient market."

Schwartz views the company's $1-per-CPU-per-hour tariff as a challenge to obfuscators such as IBM who, he says, hide the cost of computing in complex pricing structures and consulting fees. For its part, IBM argues that the $/CPU/hour metric is just too simple to be useful.

IBM's position, which maintains that not all CPU cycles are created equally, has some merit, although it is rather undermined by the company's other counter claim that, using its IBM Deep Computing Capacity on Demand grid service, it can undercut Sun's $/CPU/hour tariff by at least 30¢.

The argument over how to charge, and what to charge for, is not likely to be resolved for a long time. Until it is, true utility computing – where customers buy and sell capacity across a heterogeneous grid infrastructure – will not be possible.

Today, points out Meta Group analyst Phil Dawson, even Sun clouds matters by requiring its grid users to use its own preferred software stack. This is all very well, he says, but "if one company owns the stack, third-party apps, databases and application servers are not going to have that level of [pricing] granularity," says Dawson. Without the same transparency and simplicity in software pricing that Sun claims to offer for the underlying platform, the cost-per-CPU model is fairly redundant, says Dawson.

Many other issues must be resolved before utility grids replace today's server-centric model of computing. Potentially the most intractable is establishing the common standards needed to create an exchange that can trade resources from IBM, Sun, HP and their competitors on one floor.

To this end, the three grid pioneers have joined Intel to form the Globus Consortium, to work towards the adoption of an open standards toolkit for building and managing grids. While cynics might suggest that a simply interoperable grid is incompatible with a high-margin services business, IBM's Bricard insists: "Everyone is optimistic that it will happen. There is no doubt about it because the IT world is so complex at the moment that it is not sustainable."