The creation and distribution of business information to knowledge workers – the whole process of getting the right decision-making information, to the right people, at the right time – is fundamental to any modern organisation. Often, though, that process is hampered by an inflexible and poorly integrated information architecture, resulting in the delivery and presentation of information that is inaccurate, partial, misdirected or out-of-date – with all the negative business impact that can have.

 Despite decades of endeavour by IT organisations and their software suppliers to address this issue, there are still too many barriers to the delivery of key information to end users, barriers that include a diversity of data sources and data formats; technical and ease-of-use issues that prevent users from adding new sources of information; and a reliance on the IT organisation for changes to their information access, delivery and visualisation programs.

Data and application integration technologies have been the key mechanisms applied to this area to date. But their focus has historically either been on data warehousing, in which data is batched up and fed to a warehouse for analysis; or on application-to-application integration, where data is switched between incompatible applications.

Not only do these approaches make difficult any notion of gaining a real-time understanding of the state of the business but they are also typically restricted to dealing with the kind of structured data that fits well into the tables of a relational database.

A look at the recent survey of Information Age readers’ perceptions on the Information Access and Delivery portrayed the extent of those frustrations (see graphs through this report).

Today, there are new business pressures that suggest businesses find that far from adequate. Take the example of a pharmaceutical company where senior executive reports need to indicate progress of specific drugs trials, the cost of research, how well the organisation is meeting compliance obligations, and so on. Such a report may have to assemble and present information from half a dozen information sources – some internal, some external; some structured, some semi-structured, some unstructured.

For the past half dozen years or more, scores of vendors have been trying to solve that integration problem, often defining the challenge as enterprise information integration (EII).

One of those who set the scene for EII was Frank Gilbane, the CEO of content management analyst and publishing company The Gilbane Group.

The solution to the problem has always centred around XML, he has stated frequently. “Most business applications need to access or integrate information from multiple sources, sometimes even from outside firewalls, and this turned out to be an awful lot more difficult than most people expected, even when there is a well thought-out architecture. Beyond the notion of enterprise application integration (EAI) is a pressing need for EII that uses XML transformation technology to streamline the multi-structured integration,” he wrote in 2004.

In essence, EII seeks to deliver reusable ‘views’ of real-time business information, with the user perceiving the data as existing in a single location even though it might have been assembled from multiple, incompatible sources. Critically, all EII observers agree, that requires access to real-time data.

From the developer’s perspective, EII accelerates the creation and maintenance of many common types of application – especially those supporting decision-making, customer service, compliance and business performance management.

Business drivers

There are numerous drivers forcing companies to address this information integration challenge.

Sarbanes-Oxley and other regulatory edicts require managers to have a much greater understanding of the state of their organisation or department – and the reports they require can only be assembled by drawing on sources containing a mix of numerical data, text and multimedia data. Something as common as an end of quarter financial report is a good case in point

Beyond compliance, there are scores of other drivers: from companies managing M&A activity to manufacturers handling the constantly changing price and product data of their electronic catalogues.

“Enterprise customers are dealing with lots of issues around compliance, M&As, the adoption of new business models, and that presents a major challenge for IT organisations to try to seamlessly integrate disparate systems and provide a visual interface to that ‘mash up’ of information,” observes Kazunori Ukigawa, CEO of JustSystems, the Japanese office software giant which has crafted an XML-based information integration, applications development and visualisation suite known as xfy.

Creating integrated information views has, of course, been possible in the past – but it has usually required the commitment of significant programming resource. Moreover, a lack of flexibility with the resulting software has meant that any change the user has required when responding to shifting business conditions has had to go through the IT organisation – with the change often taking several days.

That situation has given rise to a new wave of platforms that not only present consolidated views of structured and unstructured data in a compelling way, but which ensure the end user can pull in new sources when needed, change the visual interface and still have changes in the backend data reflected in near-real-time in their report, portal or dashboard.

That real-time element is key, says Hideki Hiura, CTO and chief scientist of JustSystems’ North America subsidiary. “In the financial markets, for example, things change very fast. They are very dynamic: regulations change, companies merge. All of this [related] data would be difficult to handle with very static systems,” he says.

XML mash up

Smoothing data access to multiple sources can have a major impact on productivity. According to  Infostructure Associates, the coding of data access can account for up to 40% of a developer’s time. Another drain on time has increasingly been the effort required to bring together the multiple types of XML that are in use within many organisations.

While XML is recognised as a key mechanism of web services integration, different parts of the organisation have adopted flavours of the language designed to fulfil their specific requirements; XBRL for business reporting, ebXML for electronic trading, UBL for handling the exchange of purchase orders and invoices. There are also numerous industry-specific XML varieties: ACORD in the insurance sector, FinXML in capital markets, GeneXML in biotech, and so on.

“When you try to extract different data from different data sources you might have many kinds of XML you’ll need to deal with,” says Ukigawa. “XML is a meta-language that can define other languages, so any architecture needs to be able to handle and combine these different types of XML or XML schema.”

The current role of XML is largely limited to providing standard means of exchanging data, says Ukigawa. But there is a pressing need for XML to be a platform for freely combining the huge variety of functional XML components that are being created within organisations.

 Indeed, JustSystems predicts that business information systems environments will “rapidly shift to native XML applications in the future”. And that is already happening in areas such as financial reporting, where regulators in many European countries are prescribing the use of XBRL for the compiling and submitting of statutory reports. “XML is becoming not just an advantage but something that the enterprise is obliged to use,” says Ukigawa.