In-memory computing (IMC) has continued to rapidly evolve database management systems (DBMSs) as organisations crave the ability to run fast-changing complex transactions on large datasets.

The method of storing application data in a computer’s main DRAM – rather than the norm of electromagnetic disks – is being increasingly utilised to absorb and process data that continues to grow in not just volume, but velocity and variety too.

As organisations seek digitisation through embracing trends like the Internet of Things, mobility and social media, conventional DBMSs have proved ineffective – or, even, inhibitive – in providing the scalability and real-time analytics capabilities required.

With IMC offering an affordable answer to these challenges, Gartner estimates the market will exceed $95 billion in revenue by year-end 2018 as 50% of large organisations adopt it to enable digital business strategies.

>See also: Remember the titans: who will win the in-memory battle?

Much of this adoption, however, will be indirect exposure with IMC technologies bundled into new cloud services and traditional software products like business-process-management suites and business-intelligence tools.

Outright adoption of IMC continues to be driven by industries where scalability and fast insights are most desired, such as financial services and telecoms, but with digital agendas transcending industries, the use cases will increase in variety.

As a striking endorsement of the technology, Gartner recommends IT leaders to assess whether their current DBMS vendor can support IMC needs in a reasonable time frame – and if it doesn’t, consider alternatives.

‘Increasingly, mainstream organisations are adopting IMC for a variety of use cases, such as ecommerce, travel reservation systems, internet and mobile banking, and corporate performance management,’ says Gartner VP Massimo Pezzini, who identifies four main trends that have emerged.

Firstly, a growing number of open-source IMC technologies, as well as IMC-enabled hybrid architectures that mix transaction and analytical processing on top of the same live database.

The availability of IMC capabilities in the form of cloud services has also emerged, as has the intersection between IMC and big data technology.

‘IMC is imperative for organisations to realise their big data potential,’ says Mark Darbyshire, UK&I chief technology advisor at SAP, which has aggressively positioned its HANA platform as a leader in this field. ‘IMC helps businesses to quickly detect patterns, analyse massive data volumes on the fly, and perform their operations much more quickly than was previously possible.

‘Whether it’s hiring the right candidate from a pool of candidates or calculating insurance risk, the information that businesses require to inform the right decision is out there and, for the first time, businesses have the technological capability to do something with it.’

All or nothing

Having strongly marketed HANA, SAP is often heavily associated with IMC, but its competitors claim that CIOs could be put off by the prospect of having to rewrite applications for their database.

This is because SAP has championed an ‘all or nothing’ architecture that prevents modules from being replaced by database incumbents Oracle, Microsoft and IBM, which have instead incorporated in-memory capabilities in their flagship offerings.

According to Darbyshire, in many cases applications can simply be ported across as HANA supports standard database querying like SQL, but he pushes the case for rewriting in order to reduce the complexity of the database structure and thus simply applications.

Sean Jennings, SVP of solutions architecture at SAP partner Virtustream, explains that while translation of standard SQL queries will not offer the best possible performance, it does provide a quick-win delivery of many IMC capabilities – particularly a single platform for both transactional and analytical applications without the complexity of maintaining separate data warehouses.

Meanwhile, conventional databases that leverage in-memory technology are often evolutionary capabilities that act more like massive cache than a true IMDB architecture, he says.

‘By the very nature of their design, these legacy database products tend to lack the revolutionary architectural differentiation delivered by SAP HANA to date. This makes sense given the goals of those vendors – these are interim features on a path to a true in-memory architecture. SAP has fewer concerns regarding legacy database environments, so they can break a few eggs in creating their IMDB omelette.’

But while the most value of IMC is obtained by leveraging native IMC architectures, there can also be strong value in retrofitting existing applications.

‘Established applications are designed with the limitations and constrains of traditional technology in mind,’ says Pezzini. ‘IMC removes these limitations and makes it possible to design applications that were previously unthinkable.’

He adds that, while there is value in what other DBMS vendors are doing in terms of adds IMC capabilities to their products, some of the limitations of the traditional approach are, inevitably, still there.

IMC add-ons to existing DBMS products are a good fit for certain types of applications – like analytics – but not necessarily for others, such as transaction processing.

It should also be noted that in-memory DBMSs like HANA are not the only form of IMC technology, with products like in-memory data grids, in-memory stream analytic platforms and data visualisation tools more suitable for certain use cases

‘IMC is not only about SAP and it’s not only about in-memory DBMSs – it’s much broader than this,’ says Pezzini.

Grab the fruit

But with all these options out there, and a lot of conflicting noise being made, the question remains: how can CIOs approach the process of finding the right solution for their organisation?

Pezzini advises them to look for ‘low-hanging-fruit’ opportunities, like analytic reports that take too long to produce to the point of being almost useless, applications that are running out of stream from a performance or scalability stand point, or business users screaming because they want new analytics applications by tomorrow.

Those are situations where use of IMC technology can help effectively address the issue and deliver tangible and recognisable business value.

It may be too early to strategically endorse IMC as the data platform of choice for all kind of different applications, but it’s definitely worthwhile adopting it for selected, even business critical, projects with demanding speed, scale and real-time insights requirements.

‘For example, there are mobile app projects or Internet of Things initiatives that would be very hard, if not impossible, to implement using conventional technologies,’ he says.

Whilst revolutionary in nature, IMC can also coexist with traditional approaches and other facets of companies, like how they store data.

For example, some organisations may want to use IMC to deal with the real-time data stream analysis, but a traditional data warehouse or Hadoop platforms to support historical analysis.

>See also: If HANA fails, SAP dies: Teradata CTO

‘We expect a long-term coexistence between IMC-based and traditional architectures, which, in some cases, are still the best approach,’ says Pezzini.

Meanwhile, the market will, no doubt, continue to mature quite rapidly, having a significant impact on the design of physical servers optimised for massive memory footprints required for enterprise databases.

New products will bring enhanced enterprise capabilities around the in-memory database ecosystem, including simplified architectures and reduced costs for high-availability, synchronous replication and rapid disaster recovery.

‘We will also see a continued trend towards delivering in-memory databases on virtualised platforms and in the cloud,’ says Jennings, ‘thereby driving down the costs of delivery and consumption of high performance in-memory database.’