If you are the executive of a software company trying to get a technology ‘across the chasm’ and into the mainstream, the first principle is to talk the whole venture up. Mark Palmer, vice president and general manager of the Apama division of Progress Software, knows this well enough.

“We’re seeing 350% year-on-year growth,” he says of Apama, which develops software for analysing patterns of events in real-time data flows. “These are very exciting times.”

To back up his claims, Palmer cites a string of customers, mostly blue-chip names in financial services, as evidence that Progress – better known for database and software development tools that are embedded in applications by resellers – has staked out a leadership position in the rapidly emerging area of complex event processing (CEP).

CEP is a new category of software that evangelists of the technology say will soon play a huge role in enterprise IT. Among its celebrated enthusiasts are MIT professor Michael Stonebraker, an pioneer of relational database design, and professor David Luckham of Stanford University. Progress itself entered the market by unexpectedly buying up the British start-up, Apama, in November 2004 for $25 million.

Few experts question the thinking behind or ultimate importance of CEP. If businesses are to react to events as they happen (or even before they happen), it is faster and ultimately easier to analyse at least some of that data as it flows around the network, before it is filed away into a database.

In fact, cash-rich financial services companies have been building their own proprietary CEP-type systems for a decade or more, mostly for algorithmic (or programmed) trading.

But the technology experts – and investors in CEP – are much less sure about when today’s trickle of orders for CEP systems will become a flow. At present, the total global market is believed to be a few tens of millions of dollars.

However, Palmer of Progress believes the market is at a turning point, not just because of increased orders, but also because businesses outside the core of investment banking are set to take it up. There are signs that retailers are ready to use the technology for analysing supply chain information, and that retail banks and ecommerce companies want to use it to detect credit card fraud.

One high-profile Apama customer, BGN, in Holland, uses the system for managing its inventories of electronically tagged books.

If Palmer is right, CEP will have a fairly quick gestation. Apama was only founded in 1999, and its most direct rival, Streambase, emerged in 2003. David Luckham proposed the technology and a standard approach for analysing events just five years ago, in 2002.

The battle is now on for CEP suppliers to establish supremacy for their software. Progress is not known for building high-end systems, but the technical resilience and scalability of Apama will give it credibility, as will the fact that it is the only supplier to offer what it

considers to be the three core elements: complex event processing, business activity monitoring (BAM) and data management – the ability to trap the events and replay them back. This breadth has helped the company to win a potentially lucrative partnership with Microsoft, as part of its banking compliance systems.

But Apama is still a small unit of Progress – so small that its financial results are not yet broken out. The Enterprise Infrastructure Segment of Progress, of which Apama is only one part (along with Sonic, Actional, DataExtend and Objectstore, had revenues of $59 million in 2006, with losses of $29 million. This calibre of performance reveals that, effectively, Apama is still a start-up.

But in its favour, the company is currently faced with only light competition: Tibco, with its Business Events product, is potentially formidable, and will in the future be helped by its strong messaging technology, while Streambase, backed by Stonebraker, is also vying for early leadership in this small market.

However, as the CEP market grows, analysts expect the bigger dogs to bark louder: IBM, Cisco, Oracle and Microsoft are all known to be working on their own strategies.