The long-awaited legal ruling in the dispute between BSkyB and EDS, the IT services company now known as HP Enterprise Services, that was delivered in February 2010 in favour of the satellite broadcaster served as an eloquent reminder that it is not just government IT projects that go spectacularly wrong. The private sector, too, has its fair share of embarrassing failures.
And while it can be argued that since 2000, when the dispute began, the IT industry has smartened up its act somewhat – or at least that customers are a lot warier of their suppliers post-dotcom crash – still it could not be said that the art of IT project management has been mastered once and for all. Also in February 2010, to give but one example, car-maker Toyota admitted that a software glitch was causing certain models of its Prius range to malfunction.
Ironically, the Lean Manufacturing principles developed by Toyota, which espouse the prioritisation of customer value in manufacturing processes in order to maximise efficiency, are the ideological forebears of the most significant recent endeavour to improve the effectiveness of software development projects, namely Agile development.
Four or five years ago, Agile was the new buzzword, popular among developers and catching the attention of their corporate bosses. Today, it is approaching the mainstream, with most large organisations having some understanding of the term – accurate or otherwise.
In some cases, the adoption of Agile in an enterprise setting has been met with opposition. In others, it has become just another tool in the project management toolkit. There are some, however, that maintain that Agile, and the related field of Lean, represent an opportunity to fundamentally rethink how a business operates, an opportunity few organisations have successfully mastered.
The Agile Antichrist
The Agile framework, as applied to software programming, comprises a number of principles, among them iterative development, automated testing and continuous integration. Perhaps most important, though, is the fact that the delivery dates are fixed but the scope of a project can change. Projects will begin with a rough idea of the scope, but detailed requirements will only emerge once the project has been divided into a number of jobs, which in turn have been prioritised and scheduled for execution.
This last point, says Richard Stobart, chief executive of Agile consultancy Unboxed Consulting, is the one that large organisations often find hardest to comprehend: "People will say, ‘Surely you can’t start building before you have defined everything?’ or ‘I need to see a project plan before I’ll give you any budget.’"
Stobart’s conversion to the Agile methodology, as it happens, occurred while he was working for BSkyB, on the project that would go on to cost EDS/HP Enterprise Services £200 million in compensation (Stobart was an expert witness during the trial). Before starting his own business, he and his colleagues gradually introduced some of the principles of Agile development at BSkyB. By 2005, after it acquired Internet service provider Easynet, BSkyB was operating the largest Agile development shops in London.
In corporations, it is not project managers themselves who typically struggle with Agile methods, says Stobart, but the Project Management Office that governs all projects across the enterprise. “The PMO is the Agile antichrist,” he explains. “They have their project plans, and they want all the project managers in the organisation to report in every week on where they are according to the plan, and to have part of the design signed off.” Part of the problem, he argues, is the personnel that staff the PMO. “You often have people working in PMOs who have been formally trained in project management, but who haven’t themselves worked as project managers. This gives them a very constrained view of what needs to be done to deliver a project.”
In fact, says Stobart, when properly executed the Agile approach provides far greater governance than the so-called ‘waterfall’ approach it replaces, in which everything is delivered on one final date (although, he says, that is rarely achieved): “Once you’ve got your mind around the fact that you’re delivering iterations, and that the scope itself is flexible, Agile is much more controlled than a waterfall project.
“Let’s say there is a year’s worth of work to do, starting on 1 January. In the waterfall approach, 20% of the time is requirements capture, 20% is design, 30% is development and the last 10% is roll-out. By the time you’ve got to mid-March, you haven’t written a line of code but you’ve got 1,000 pages of requirements.
“How do you know whether you can do that in the 30% time allotted to coding? You don’t, so the developers guess it will take them six months but the project manager then says they’ve only got three months. Then the enterprise architects do the design, and halfway through they might find there is a flaw in the requirements. If you are lucky enough to have started any coding by then, you might have to scrap it all as a result.
“In the Agile approach, you spend some time with a business person who has a strong idea of what they want. They can’t quite envisage it precisely, but they can tell you what they want and what the most important things to achieve during the year are. The architects and designers can tell you what the biggest risks are. By the end of January you have a broad scope document, telling you what it is they want delivered, what the risks to the project are and what technology will be used; and you are in a position to identify what should be delivered first, or which risks need to be addressed first.
“Now the PMO might say at this point that we are behind and that a waterfall project would be further along the scoping process by then,” continues Stobart. “But there is no guarantee that anything they’ve done so far will work, and you won’t find out whether it will until much later.
“Agile projects that fail, fail fast, which is a huge advantage. You want to know as soon as possible if a project is not going to work, whether its because your biggest risk is too great to overcome, or because the top business priority is impossible to achieve.”
Cleaning Up
Done badly, however, Agile can be disastrous. “We do a lot of business in what I’d call Agile recoveries,” says Simon Bennett, head of Agile, Lean and strategic thinking at EMC Consulting, formerly Conchango. “When someone has gone in and made an absolute dog’s breakfast of a project, a department or even a whole organisation in the name of Agile, it can take a lot of work to put right.”
The reason this occurs, he says, is that demand for Agile is outstripping supply, which has resulted in a dilution in the quality of the average Agile consultant or trainer. “For every person who has been doing it for a long time, and so understands the systems thinking, the complexity science and the human psychology behind Agile, there are another 100 who’ve taken two-day courses and become Agile coaches because they don’t know what else to do with their lives.”
Nevertheless, the fact that demand is so high demonstrates the value of Agile done right, Bennett says. And today, that demand is not just for Agile software development or Agile project management, but also for Agile business strategy. “A lot of our work is now about applying Agile at a ‘macro’ level, about transforming an entire organisation,” he says. So what is an Agile organisation? Bennett’s point of reference is Apple Inc. “The rise of the iPhone is a case study for what Agile could do for your organisation,” he explains. “It is not just a product, it is a whole value stream, with iTunes and the App Store behind it. And in fact, the iPhone came out of the iPad project, which had been going on for years. On the way to developing a touch- screen tablet PC, they realised they could build the iPhone and the iPod Touch – this very much reflects the Agile principle of the ‘Art of the Possible’.”
Bennett warns against the hope that adopting Agile in software development will result in an Agile epiphany for the business. “Even if you manage to get an 800% efficiency improvement on a seven-man software development team, the effect on the bottom line for a corporation is almost non-existent,” he says. When developers try to push Agile across divisions, he says, “they get beaten down by the technical governance people, or the project governance or finance people, and they don’t know what hit them”.
Instead, as ever, enterprise-wide Agile requires board-level buy-in. “You can’t go from bottom up,” says Bennett. “That’s not how a large corporation works. The first thing I do with a new client is book a meeting with the project management officer, and then with the finance director.”
In The Toolkit
Chris Field, president of the Project Management Institute, agrees that for the benefits of Agile to be felt, an organisational transformation is often necessary. “It requires a change in mindset, especially if you’ve got an extremely prescriptive set of governance processes,” he says.
However, Field does not see Agile as a special case in the project management toolkit, but believes instead that it is, like all other approaches, a useful tool in the right circumstances.
At the time he spoke to Information Age, Field was working on what he describes as a ‘pseudo-Agile’ project, in which Agile and waterfall are used in conjunction. “The project involves a number of different applications,” he explains. “When we’re working with off-the-shelf packages (where we’re mainly doing configuration work), that lends itself to Agile, whereas in the case of the more monolithic applications built in COBOL and the like, we have to identify the requirements upfront.”
Combining the two approaches has called for a hybrid methodology to be drawn up, and while that does take some getting used to, Field says, as long as the methodology is well documented it can be followed just like any other.
Field says that IT project managers are now less concerned with methodological purism than they once were, and are instead interested in more practical considerations. “The way to look at these things is to think about the characteristics of the project, and use different aspects of the methodologies that are appropriate,” he says. “Going forward, Agile will drop into the same toolkit as past approaches like waterfall and Prince.”
Nevertheless, Agile is an approach well suited to the uncertain business climate, he says: “Right now, project managers are interested in any tools that help them to avoid any surprises, and Agile is one way of doing that.”
Agile, then, is becoming part of the furniture when it comes to IT project management. But if the benefits to speed and transparency are to be enjoyed by the organisation as a whole, businesses cannot hope that Agile will just spread across divisions organically, such are the cultural barriers in the typical enterprise.
