Elaborate claims are made for enterprise application integration (EAI) products. The technology, say suppliers, will enable disparate business applications to interoperate seamlessly, exchanging data regardless of its format. In this way, they claim, it creates corporate efficiencies and generates substantial cost savings.

But EAI software is also expensive and time-consuming to implement – which is why, in a difficult economic climate, suppliers are increasingly called upon to back up their claims with substantial evidence that prospective customers will see a return on their investment.

In response, many EAI suppliers now use return on investment (ROI) calculators to generate figures to assist them in closing a deal. How realistic are their projections? And how much faith should organisations invest in them?

"ROI calculators allow businesses to take the guesswork out of forecasting returns on their new projects," claims Alun Baker, UK managing director of EAI vendor Tibco. "This scientific approach helps to set measurable success criteria and ensure that only the most valuable projects for the business go ahead," he adds.

But this "scientific approach" comes with considerable complexities, says Karsten Alva-Jorgensen, an associate partner for consultancy Accenture. "It can be challenging – often the business case [for an integration project] has not been well documented," he explains. "You want to improve the total cost of ownership of systems, but that really requires a quantifiable view of cost. To show what ROI you are likely to achieve, you have to know how much [your enterprise systems] cost today – including intangibles," he says.

So how are ROI projections generated? Many consultancies and auditors use an ROI calculator to define the data needed to determine possible paybacks on projects. By drilling down into specific business processes, the calculator generates metrics for improvements to the company. Order processing, for example, may be analysed using data such as percentage of orders processed using manual means, while time-to-market analysis may look at the gross profit per day of a product.

An ROI study should not be a drawn-out process, says a recent research report from IT services company Compass-I. However, to be reasonably realistic, it does need to be based on accurate and comprehensive data. This may be a problem for some organisations; indeed, they may be embarking on an integration project specifically because managers find it difficult to access accurate and comprehensive data.

Nevertheless, say Compass-I executives, an ROI exercise can create vital project momentum within an organisation. "Working through a scientific and measurable process to identify potential returns will create a much greater level of confidence in the initiative and help reduce the risk to the business," they say.

But technology will not create benefit unless it is implemented in tandem with fundamental business changes, warn analysts. "Measuring ROI provides no benefit in itself," argues Nigel Hughes, head of ebusiness at Compass Management Consultancy. Dan Merriman of analysis company Giga Group agrees.

"The increased focus on ROI has improved IT's ability to select projects that have the best chance of providing value to the corporation. Yet it does little to help IT actually deliver that value," he says. "IT needs to work closely with the business to identify and implement success metrics for major IT projects. Business benefits typically don't result from technology alone, but from a complex integrated effort that includes business process improvement and change management."

Alva-Jorgensen says that many of the questions he gets asked are about re-engineering business processes and how to quantify them. "We ask what the business metrics for that process are, model that, then suggest common metrics to use. Then we work to agree on those."

At analyst group IDC, senior project manager Andrew Walton and project manager Niccola Urbani have been working on EAI ROI analyses within the company's consultancy division. They determined some strict parameters for examining ROI. "We looked at networks prior to application integration done by clients and the number of interfaces that existed in the legacy systems. Then we compared that to the number of connections after EAI," explains Walton.

Urbani compares the two situations to people sitting round a table. Either they can all try to hold hands with each other to link up or they can grab a pole in the centre. The former situation is analogous to a legacy system that interfaces with several other applications; the latter is after the EAI process, when the applications all have interfaces to a single, central information conduit.

Not only does the number of interfaces increase less quickly as new applications are added to the system (one new interface for each new application compared with one fewer than the number of applications in the legacy systems), says Urbani, but information propagates far more quickly through the enterprise. Systems maintenance costs, development costs and information bottlenecks are all reduced.

Walton and Urbani claim their calculations show that the ROI over three years for a typical company implementing an EAI product is 130%, and projects typically pay for themselves after an average of 8.7 months. Alva-Jorgensen tells his clients that they can expect to make their investment back and achieve savings of a further 20% using EAI. "We do try to be conservative, and clients have been pleased by the results," he says.

Hit or miss?
But is ROI the only measure of a successful EAI implementation? Martin Anderson, a director of systems integration specialist Rubus, thinks not. "Trying to establish good metrics for cost-benefit analysis can be very hard in infrastructure projects, since almost by definition, once you have started doing things the new way, there is no old way against which to compare it. The real benefit is not in the quantifiable cost savings alone. It is much more in the less quantifiable increase in flexibility and responsiveness, which can have dramatic effects on business competitiveness."

And unexpected ‘hiccups' have a habit of de-railing integration projects. Louis Vintro, director of Vitria, argues that the problems involved in EAI mean that companies frequently fail to achieve predicted savings: "It can be difficult integrating, automating and managing cross-application and cross-functional work processes and there is a general lack of clear data exchange standards. While a company that uses off-the-shelf software and does not want or need to integrate with the application infrastructures of their trading partners will be able to achieve the predicted savings, companies that mix-and-match EAI suppliers, have bespoke software, or who want to collaborate with other companies across the web using XML or via EDI, will find achieving those levels of return hard."

By contrast, IDC analyst Walton admits that integration adapters for legacy applications "tend to be a little more expensive", but says that even where companies have many legacy applications, the company still saves money.

"Integration is still a complex and largely a bespoke activity," maintains Tony Christian, president of systems integration company Aveva Consulting. "The ebusiness industry has been mixed in terms of its integration experience. On the downside, it has been hindered by the requirement to integrate immature, even prototype systems, because of the importance of time-to-market. However, on the upside, it has produced technology and standards like XML that have enabled integration scenarios that were simply too difficult to achieve before."

Christian says that a company considering an integration project needs to establish the purpose of linking applications together, and determine what business benefits it expects to gain. "Organisations should set goals to achieve and those goals should not be technology-focused, but business-related."

Furthermore, organisations need to carefully consider their future integration needs. Anderson of Rubus says a common mistake made by companies when considering how to integrate applications is to simply look at the "here and now". "Integrating existing applications can be tedious, frustrating and time consuming, but it is not impossible. Harder will be creating an architecture where when the next integration challenge comes along, the process has already been dramatically simplified. There's no point in swallowing up the whole resources of your IT team without making sure that the process of integrating further applications in the future will be a whole lot easier."

"What we've found," concludes Alva-Jorgensen, "is that it really comes down to change management. There needs to be enough focus on the education and training involved to understand how to apply the technology. Changes to business processes will occur throughout the organisation and across traditional organisational barriers – and people need to know how to cope with that."