In today’s always-on, hyper-connected digital economy, consumers and employees have come to expect a constant flow of tech innovation from the organisations that are striving to meet their needs

However, at the same time, they also expect continuity. The digital services they rely on; such as online banking, e-commerce sites and instant messaging applications, are expected to work seamlessly and deliver a perfect user-experience every time.

Modern technologies and digital architectures are critical to organisations’ ability to meet these needs and expectations, so it’s no surprise to see so many investing so heavily in microservices, containers and multi-cloud services. Showing the scale of this investment, analyst IDC forecasts spending on digital transformation technologies will rise to $2 trillion by 2020.

The hidden cost of innovation

However, such rapid technology adoption comes at a higher price than simple dollar values. In recent years, technology environments have become vastly more complex. For example, organisations have transitioned from physical servers, to virtualisation, before later migrating services to the cloud.

>See also: Legacy to cloud: former founder and CEO challenges old business

This trend has in turn led to the growing use of multi-cloud, as organisations look to create best-of-breed IT ecosystems tailored to their own specific requirements. Now, CIOs are realising that their legacy applications weren’t designed to run in these environments, so they’re rebuilding them in cloud native architectures, led by the use of microservices and containers.

These modern approaches are critical for organisations to maintain the speed and agility needed to keep up with the new ‘born digital’ start-ups that are challenging them for market share. However, such rapid changes have created hyper-scale, hyper-dynamic and hyper-complex IT ecosystems that are spiralling out of control. Recent research released by Dynatrace revealed that the average mobile or web transaction now crosses 35 different technology systems or components, compared to 22 just five years ago.

This provides a stark illustration of just how quickly complexity is growing. In these new environments, it is virtually impossible for IT teams to maintain end-to-end visibility of application performance, putting their ability to manage the customer and end-user experience at risk.

The cloud complexity conundrum

Chief amongst the woes of modern organisations is the complexity that comes from the continued transition towards the cloud. First, as they look to use services from a multitude of different cloud providers such as AWS, Azure, Cloud Foundry and SAP amongst others, organisations must also be prepared to manage each of those services.

Often, that means finding monitoring solutions that can integrate with the specific requirements of each of those environments and then instrumenting them manually, which takes a great deal of time and effort for IT teams.

>See also: Cloud growth brings ERP integration challenges

Adding further complexity is the shift towards cloud native architectures, where applications are broken down into microservices and run in containers. Whilst this approach does significantly increase agility and help organisations to maximise the benefits of moving to the cloud in the first place, it also makes it difficult to manage the overall performance of a digital service.

In fact, the Dynatrace study found that over three-quarters of CIOs feel it could become impossible to manage digital performance efficiently as IT complexity continues to soar. The challenge isn’t just seeing inside and monitoring these new architectures, it’s also being able to keep up with the constant changes that are inherent to these hyper-dynamic environments.

The business and innovation drain

However, the impact of this hyper-complexity isn’t just a challenge for IT; it has serious implications across the wider company. A consequence of digital services becoming ever more business critical is that IT teams are required to provide answers to an increasing number of organisational stakeholders; from the CEO down to front-line customer services teams.

As such, IT now has to identify anything from why a particular region or outlet experienced a dip in sales, to the impact that process changes introduced by the compliance team will have on the wider organisation.

>See also: Multi-cloud and application services are fuelling digital transformation

Unfortunately, it is becoming harder than ever for IT to find the time and resources to answer these questions effectively, as it becomes a growing struggle to manage performance amidst the mounting complexity of their rapidly evolving cloud native environments.

The combination of these pressures is undoubtedly creating a major drain on innovation, as day-to-day operations become a black hole that sucks resources away from more strategic IT projects that are vital to the business’ ability to stay ahead of its competitors.

A hyper-smart solution to a hyper-complex problem

Ultimately, it will be impossible for IT teams to overcome this challenge if they continue to rely on tried-and-tested, conventional methods of digital performance management. In the face of hyper-scale, hyper-dynamic environments, human operators alone are unable to sift through the enormous volume of performance data that offers the crucial insight into the health of their digital services. It’s no wonder then that four in five CIOs think that artificial intelligence (AI) capabilities will be the key to their ability to master IT complexity.

>See also: Painting a multi-cloud masterpiece

In an increasingly cloud-centric future, AI provides a much-needed lifeline for IT departments, enabling them to instantly analyse and understand the millions of interdependencies that exist between their cloud native applications and the hyper-complex multi-cloud infrastructure that underpins them. This capability will be vital to IT’s ability to maintain end-to-end visibility across the full stack of IT and correlate that with its impact on the customer and user-experience.

Ultimately, it’s important to realise that next-gen digital ecosystems come with next-gen challenges that must be overcome before they deliver on their promise. The rapid advances in AI capabilities that have been seen recently couldn’t have come at a more pivotal point in our digital evolution and will be a major game-changer for performance management in our modern cloud native world.

Sourced by Matthias Scharer, VP of business operations, Dynatrace