Has the IT industry spent thirty five years setting itself up to constantly fail? It is an uncomfortable question, but one that has some validity, judging from the spate of publicly documented project collapses and overruns that emerged in 2004.

Despite the establishment of a whole sub-industry devoted to project methodologies and with the shelves of IT departments around the world groaning with manuals on how to bring projects in on time, within budget and to specification, IT has consistently failed to deliver to these aims (see box, ‘The state of things’).

Maybe it is over-optimism on the part of project leaders; maybe it is because parts of the software development process are more art than science; perhaps it is because the processes of a business are too dynamic to be hard-coded in software.

Whatever the reason, IT management in recent years has realised their portfolio of on-going projects has simply become too complex to be delivered using historical approaches. And, many would admit, those historical processes have been sorely lacking.

Moreover, the task has become complex as IT development has become a matrix of activity. While traditional development tasks might all have been handled (and therefore controlled and resourced) internally, that is becoming less common.

Just like manufacturers before them, IT organisations, under pressure to deliver high quality products at lower cost, are looking to distribute the task over a mix of internal and external resources. A typical scenario might be that the business requirement and IT’s role in meeting that will be specified by the organisation’s own IT management in conjunction with the business. The development could, however, involve a team from an IT services partner working on-site, defining requirements capture or managing user training, with another team coding the software off-site, and yet a further group at a near-shore or offshore facility testing the software.

Distributed change

That multidimensional effort is difficult enough to manage if the project is static. But one of the key reasons for the failure of projects is that IT often struggles to deal with the dynamic requirements of the business. And accepting that such changes need to ripple through a tier of three or four groups is a new kind of challenge.

“In a multi-sourced environment, organisations need to create virtual teams,” says Christopher Lochhead, chief marketing officer at governance and applications delivery software vendor Mercury Interactive. “The future role of the IT director is really one of managing centres of excellence distributed across the IT department and outsourcers.”

What is also clear is that current practices and tools are not up to that job. Many organisations with hundreds of programmers and scores of projects underway at any one time are managing them with little more than the help of Excel spreadsheets, Gantt charts, white boards and a project leader’s mental map of skills and availability.

“Businesses are running blind,” says Eugene Blaine, CEO of project and business performance software vendor Atlantic Global. “Companies are using electronic Gantt charts, but these aren’t designed for IT project management, they’re designed for building bridges. Moreover, everyone struggles to keep them up to date and therefore no one trusts them.” “Businesses are also using project management tools that have no resource management capability and no definition of milestones – just a list of deadlines,” he adds.

The sense that IT needs to do something about that is undeniable, says Scott Phares, senior vice president of solutions management at project portfolio management software company Business Engine. “A year ago, it was clear that customers weren’t ready for a really sophisticated product. But they now all see the need for a mechanism that prioritises projects and matches resources to the delivery of those.”

IT for IT

IT’s own requirements definition is pretty clear. It encompasses a broad set of applications, including:

• demand management (for prioritising projects and eliminating any duplication of effort within the projects);
• resource management (for optimally allocating from the skills pool);
• budget management;
• timesheet and expense tracking;
• risk management (for assessing the impact of missing deadlines or scaling back the scope of the project);
• contractor management.

Sitting above these, there needs to be a mechanism to create real-time visibility into the underlying variables that can demonstrate how well they are meeting a group of clearly defined milestones, with different views and different milestones for the different participants.

As such, project management tools should not be for project managers alone.

“The biggest consumers of project management should not necessarily be project leaders,” says Mike Shomberg, vice president of marketing at project management software vendor Primavera. “CIOs, CFOs and others want easy-to-interpret, graphical, high-level views of what’s going on.” That should extend right down to those executing the project, the programmers and testers, he says, so that they can see how their project is progressing, which areas need more focus, and what jobs and projects are coming up that fit their skills or aspirations.

As that suggests, project management needs to be tightly integrated with several related areas of business software.

Multi-discipline

There is a clear overlap with human resources (HR) software, with the HR department playing a key role in tracking skills availability, skills development, employee performance and career development.

Crucially, there is also a need to bring elements of change management software into the mix, so that one of the biggest contributors to project failure – an inability to absorb the naturally shifting requirements of business – are factored into the project.

“IT has to stop complaining about the goal posts moving and factor that in to project management,” says Ayman Gabarin, vice president of IT governance at Compuware.

Given the breadth of the requirement, it is not surprising to see software vendors from many different disciplines attacking the problem, often from different angles.

There are traditional project management companies such as Primavera, Business Engine and SystemCorp (recently acquired by IBM) which have evolved to become broad project portfolio management software vendors; companies such as Niku that initially developed tools for professional services groups but which have since re-oriented their focus around IT project issues; vendors with a pedigree in software quality and performance, such as Mercury Interactive and Compuware which view good ‘IT governance’ resulting from all aspects of application delivery, from definition and coding to management; and, lastly, relative newcomers such as Atlantic Global, which are evolving to provide performance visibility capabilities on top of standard project management applications.

And alongside all those is Microsoft with MS Project and Project Server. While not wanting to dismiss the Microsoft product line, many of these vendors argue that its limitations in dealing with multiple projects still leave plenty of space for others to deliver to what has become a yawning gap in the project management market.

INTEROPERABILITY

The lack of a common approach to building project management software potentially creates problems for the users. If organisations are to apply technology to the distributed delivery of projects, then they will need some level of standardisation.

For example, a CIO with a major project underway across both in-house and outsourced resources will want to be able to look into a project dashboard and see how the total effort is doing.

If external partners are using a different project portfolio management package (as is likely) then the only way that the dashboard is going to be populated with valid data is if it is extracted from one project management tool and fed into another. And that will require standards.

In terms of providing that ability for different project management tools to talk to each other, the sector is “very, very early in the discussions,” says Chris Lochhead of Mercury. “There is not much there yet.”

There is perhaps still time for that to happen. Most IT organisations are only just starting to apply IT in a sophisticated way to the management of projects — typically, as pressure is piled on them from other parts of the business to ensure better governance, to reduce risk, to deliver more consistently and to manage budgets more tightly.

At one time, that might have been possible without the application of project portfolio management, but in today’s distributed development environment, the spreadsheets and wall charts are simply perpetuating a culture of failure and blame.