Making your company more “data-driven” is a goal that everyone seems to be embarking on. McKinsey found that more than 50% of CEOs consider themselves as leading their companies’ work around analytics and data. But there is a difference between the traditional company approach where data was gathered at a regular pace and analysed periodically, and those new companies that make data part of their everyday processes. The volume of data coming through, and how quickly it can be transformed from raw inputs into something tangible, useful and actionable within your business, are the signifiers for success in this new way of running operations.
Under this big vision, there is still a degree of separation between IT and the wider business around how data gets used. The most immediate cause for concern is when teams have siloed data. In their day-to-day roles, each department can have their own tools and their own set of data. The security team uses their tools to look at data from applications and cloud platforms; the software developer team uses log data from their tools as their applications go through the software development life cycle, and the IT operations team will have data on how those applications and services run in production from their cloud platforms as well. That can lead to multiple sets of data that should all tell you the same things, but they are spread out and no-one can collaborate effectively. Worse still, you are paying for three sets of data that are essentially and functionally the same.
Bridging the gap between data engineers and business analysts
According to 451 Research, 40% of companies had more than eleven different monitoring tools in their environments, while 29% had between seven and eleven solutions. That is a large number of tools and services in place that can all be doing the same thing. Not only does this end up harming how teams collaborate, but it drives up cost around data initiatives and affects how successful those projects are over time. This can contribute to data projects not meeting their required goals – for example, Gartner predicted by 2022 that only 20% of analytics projects will deliver on their business outcomes.
The divide has likely worsened during the global pandemic too – while individuals can collaborate with their direct contacts, it is not as easy to get together with cross-functional teams to work on projects. Without this ability to collaborate, it is harder to get proper resolutions in place. It’s also harder for teams working in segregated silos to progress and get over those barriers to change. Those fleeting interactions can be very powerful at harnessing collaboration, but the move to remote working has limited our ability to do this and likely pushed us all further into our own operational silos.
As well as reducing opportunities for spontaneous communication, studies show other effects too, although these differ between organisations. For example, research from Digiterre also found that a lack of access to business experts has slowed software development. Similarly, a report from Microsoft and the University of Victoria found that productivity has not changed or has even improved. For those that want to concentrate on their own tasks, remote working has helped them focus. For those that thrive on collaboration, it has been harder to stay productive.
How to break down team and department silos for digital transformation
The speed at which businesses are able to react, and the agility with which they adapt, determines whether they survive or die. Data can influence all of this, as long as we are able to break the silos between teams and the associated tool sprawl, to drive better business decisions. In practice, this involves using the enormous volumes of data generated by digital operations and handing that information over to each team for them to use as part of their processes. Rather than each team running their own tools and sets of data, the data comes first and the central service delivers the right dashboards or analytics results for each team, whether they are in development, security or IT operations. More importantly, this process has to be carried out in real time and provide concrete recommendations tailored to how each team works. Gartner terms this process continuous intelligence, as it is based on that continuous flow of data coming through business processes.
This approach – to centralise data handling and analysis, and then get more teams involved around that one set of data – should help companies make more of their data with different teams. As an example, one such use case might be within a retailer running an e-commerce website. The business operations team may get an alert and notice a high number of cart abandonments are taking place on the website, meaning people aren’t completing their intended purchase and the company is missing out on revenue. This affects the business, so there is pressure to investigate why it’s happening and fix the problem.
That same set of observability data can be used by the development team to identify where the problem is coming from in the IT infrastructure and what is causing the abandonment issue, such as slow page load times or a bug in the checkout process. Using this data, the software development team can then put together and apply a fix and then monitor traffic and experience over time. Equally, if the issue is due to a security problem, then the security and compliance team can step in to solve the problem.
Using data over time, this can provide an ongoing record of your changes and the impact they had. For instance, you can directly compare revenue performance against any development changes you make, to help you optimise and build a more profitable business over time. Similarly, you can see where problems are starting to affect performance and take proactive steps before things get too serious.
The key is to start with eradicating data silos. This is necessary to ensure that all teams are working from the same data source, to minimise conflicting information and reduce costs. Using a single continuous intelligence platform can provide actionable data across the business. If everyone is working from the same data, it will encourage teams to work more closely together, even if they are still separated by physical distance. With more company leadership attention on the results of analytics, this should provide better results.
