Back in 2001, when the wave of hype about business process management (BPM) was first building, the technology was viewed by many potential customers – and many observers – with deep suspicion.

There were good reasons for this. BPM was just one of perhaps a dozen new software technologies that corporate buyers were being told was revolutionary – and many of them were visibly and expensively failing.

Some grand and ambitious claims were also being made by its most vociferous proponents. Howard Smith, the chief technology officer of Computer Sciences Corp in Europe, Peter Fingar, a business books author and former software engineer, and Ismael Ghalimi, the founder of BPM start-up Intalio, among others from the BPM camp, all spoke of the end of programming and a new age of software in which business processes were managed as data. They envisaged the creation of new business process management systems (BPMS) that would, as a technology, have the same impact as database management systems. And while such claims have never been discredited, they sounded highly ambitious – then as now.

The BPM sector is still immature. Products vary widely in function and the terms used to describe their capabilities.

Analyst advise that, in addition to function, customers should examine a supplier's credibility and financial stability; its customer service and support; the availability of integration tools and expertise; and, critically, its adherence to and planned adoption of standards – especially those relating to web services and process modelling.

The key elements of a typical BPM package include

• Process definition or modelling tools for mapping out and defining business processes. These should be graphical, intuitive and useful to business (rather than specialist) users.
• A process ‘engine' – a key component is a database that holds and applies all the rules and workflows set out in the process definition tool. The execution engine automatically initiates, orchestrates and manages the processes that are ultimately carried out by the organisation's applications – or manually by employees and partners.
• Tools to enable managers and staff to interact with the process engine, in much the same way that an application might. This is important for supporting manual processes, or for intervening in automated processes.
• Monitoring, reporting and analysis tools, to enable managers to see where there are bottlenecks, for example, and to monitor work in process. Increasingly, integration vendors such as webMethods and Tibco are tightly linking this to their underlying integration hubs, providing ‘business activity monitoring'
• An integration layer that links into the relevant underlying business applications – not just major packages from companies such as SAP or Oracle, but also legacy systems, such as IBM's CICS progams, and other applications developed in-house. In this way, processes can make use of underlying function that is already there. Increasingly, there will be a need to integrate and/or orchestrate web services as well as applications.

Perhaps most crucially, neither the big software suppliers nor most of the major services companies placed any significant early bets on the technology; indeed, with their heavy commitment to existing products, it was widely thought that it was not in their interest to support the BPM vision. Without IBM, Microsoft, Oracle and SAP the BPM movement lacked momentum – notwithstanding the contributions from many start-ups and smaller suppliers.

But over the past two years, the importance and positioning of BPM has changed dramatically. Increasingly, it is now being seen as an important and powerful technology – and one capable of delivering, in some cases, a dramatic return on investment. In this sense, at least, the optimism of some of the evangelists – and of a few prominent analysts (including some at Gartner, the leading advisory company) – has been vindicated.

Convergent paths

The growing importance of BPM can be put down to two main factors. The first reason is largely technical, and has to do with the convergence of three technologies: enterprise application integration (EAI), web services, and business process modelling and workflow.

Around the year 2000, almost all large organisations were coming under intense pressure to integrate their core business applications. At the same time, a new much easier method for linking applications was under development – web services.

Although even today, web services has so far only played a minor role in application integration, it became clear to the major suppliers of integration software that they needed to move away from their proprietary tools for integrating basic applications – with or without web services, that business was starting to commoditise.

Instead, the integration software suppliers needed to offer tools for defining, interlinking and managing services, business processes and workflows. As a result, BPM – previously seen as a niche modelling technology – suddenly became far more important to this group of integrating vendors, which included Tibco, webMethods, SeeBeyond and Vitria.

BPM also provides an important element that was largely missing from EAI-based solutions: the products cater for human involvement in business processes, for example by allowing for longer gaps between steps and actions, by providing prompts, screens and messages, and by facilitating and supporting human decision-making.

It is not just the integration vendors who have been alerted to BPM. At the same time that the integration suppliers were seeing their opportunity, it also became clear to the bigger ‘platform vendors' and software companies – among them IBM, Microsoft, SAP, Oracle and BEA – that applications in future will increasingly consist of services, linked together to form processes. They needed, and to an extent still need, more tools for defining processes and managing services. BPM provides at least a partial answer.

Taken together, all of these vendors have adopted many of the principles and technologies of BPM (see box), although not everyone has necessarily subscribed to the more revolutionary vision, which proposes using a new kind of mathematics (Pi calculus) and a new set of notation and process-handling standards, to provide easy, open, interoperable ‘plug and play' handling of business processes.

The second main driver for the recent uptake of BPM has come from the customer. While the vendors were thinking strategically about opportunities, threats and positioning, customers were facing a different kind of problem: they needed much greater flexibility in the development and use of complex software. BPM, although still a nascent technology, was intended exactly for this.

Gartner analyst Jim Sinur suggests that BPM can provide a good return on investment if it is used , for example, to build business processes faster (by linking to underlying applications); to reduce friction during mergers; to more easily accommodate the outsourcing of lower level business processes; and to better manage supply chains and ‘process joins' with other organisations.

Shashin Shah, a director in the technology practice of consulting company, Deloitte, says that this need for business-driven flexibility has led to a substantially increased take up of BPM since 2003. Customers, for example, use it to plan and build processes to comply with legislation; to meet requirements for auditing and planning; and to build new processes without having to change or replace underlying legacy systems.

A third reason has also helped drive the uptake of BPM: it works. Most suppliers can now cite a list of big-name customers who have used BPM successfully for at least one, and often several projects. In many instances, and especially where cross-departmental systems and human actions have been integrated into a clearly managed process, the return on investment has been very high. In these instances, says Shah of Deloitte, the return is "60% process improvement and 40% technology". In most cases, these projects are departmental, rather than at an organisational level, but enterprise-wide BPM, believes Shah, will happen eventually.

According to Gartner, BPM is becoming established as a key technology in the modern software infrastructure, even among customers that are not notably process driven. It is technology that chimes with overriding management concerns: To achieve greater agility for the business, lower IT costs, and greater control over software development and management.