The computing world has seen a significant shift over the last 15 years.

The monolithic world of IT has moved to a more services based paradigm and with it, massive changes in the way applications and IT services are both architected and ultimately delivered has followed.

Software went from mostly bespoke off-the-shelf elements to smaller highly custom distributed applications integrated via the network in order to form much more scalable offerings.

Today, a typical large enterprise may have hundreds of different applications each with a broad set of dependencies to each other along with network, storage, and increasingly cloud-based interconnections.

The resulting benefit is an incredibly agile IT environment, yet with this level of integration comes with the consequence of more complexity. So when things go wrong, unlike the days of the single monolithic application, trying to pin down the root cause and fix the problem can become a challenging task.

Finding the needle

Take for example an application slowdown. First thoughts may be the issue is due to network bandwidth or latency.

>See also: Top 8 trends for big data in 2016

However, it could be a fundamental problem within the application delivery chain, such the application’s code, with the databases, a physical hardware problem within the server, or a halt within a dependent service.

Even a minor configuration change or patch may have altered a setting that is leading to a performance hit. If the application has several dozen dependencies that may impact performance, how do you find the cause?

Since 1980’s the first port of call for IT admins on a quest to troubleshoot problems has been event logs. These files generated by applications and hardware elements including storage, networking devices and even clients collate diagnostic data that can provide clues.

Logs are just a snapshot in time on that one element and have no context around what else in happening across the environment.

IT administrators attempted to correlate logs based on time stamps, and then piece together the relationships and data flows between the different dependent elements within a workflow.

With networks at gigabit speeds, trying to pin down a problem using just logs is hard work and in some cases simply impossible.

As environments have become more interconnected, this approach is almost untenable due to sheer volume and complexity of these log files which significantly lack standards between applications, vendor or technologies.

Another method is to place physical devices, such as probes or agents, on to servers to try and sample the state of applications and connected resources.

This use of agents when an IT department had single monolithic application servers and owned all the related elements was useful but incomplete.

>See also: How to understand and fight the four faces of payment fraud

The main issue was that the agents were not able to reside on network, storage or other hardware appliances and thus gave only a partial few. Also, placing an agent on a highly tuned server, or even rapidly spun up virtual server, adds another layer of complexity to an already complex environment.

Wire data analysis

Around 2010, organisations started experimenting with the notion of wire data.

In simple terms, wire data analysis examines the flow of IP packets between applications, storage and network elements to build an accurate real-time picture of both the dependencies and communication flows across the application delivery chain that makes up any IT service.

Because the analysis takes place on the network, it delivers application-level insights without the need for agents on any device without impacting performance.

In recent years, innovators have taken this basic IP packet analysis, turned this largely unstructured data into structured and indexed repositories and added real intelligence that provides a much deeper level of inspection.

It is vital to understand; wire data is not just network data. It instead describes the fundamental ICT relationships plus a decoded view of the application specific communication protocols to deliver a granular level of insight around the transactional processes that reside within the raw data.

An analogy might be the difference between reading a line of text as one letter at a time or processing the whole sentence, within the context of a paragraph to understand the true meaning.

Wire data is, essentially, a source of truth as it is not dependent on sampling from an individual server, but is instead a picture of what is actually happening across the entire IT infrastructure at a workflow level.

This “captured” wire data is also archived to allow detailed forensic investigation and supplemented by additional data points including event logs, device polling using technologies like SNMP and even agent data, if available, to build an even more accurate model.

As wire data appliances can reside within the cloud and on third party networks, even highly distributed IT infrastructure exists in this almost living representation of an entire IT ecosystem.

Beyond break-fix

With this drove of accurate, real-time insight, the fun really begins.

Through advanced Intelligence and automation, wire data analytics systems can automatically spot anomalies from the baseline normally and pre-emptively alert administrators to potential problems.

>See also: How wire data will unlock new insight into people, applications and the Internet of Things

If a problem occurs, IT admins trying to solve issues can start to drill down into more detail, even down to individual transactions or IP packets, to find out where the problems really reside.

This can be both real-time and historical data across physical servers, databases, ERP systems, VoIP, web services, SANs, authentication servers, – in fact anything that is on or communicates via the IP network.

However, this is not a magic bullet as IT admins still need a level of competency to interpret what wire data analytics is showing them, but it is a sea change from the bad old days of logs and software agents.

As Gartner analysts, Vivek Bhalla and Will Cappelli recently identified, “Wire data radically rethought and used in new ways …will prove to be the most critical source of availability and performance management over the next five years.”

It’s not just in the arena of troubleshooting, some organisations that have been early advocates of wire data analytics are using it for security monitoring, performance optimisation, even as a method of delivering against service level agreements.

Sourced by Isaac Roybal, director of product marketing at ExtraHop