According to Big Data LDN and Cloudera‘s survey, The Fourth Industrial Revolution, data visualisation remains the biggest barrier to achieving business goals for 45% of data leaders, but is closely followed by the inability to understand the business impact of insights (34%) and the usage of AI and machine learning by UK organisations has decreased 21% in the last year.
What is going on? How can CIOs and data leaders overcome this data visualisation hurdle and why has the use of AI (and ML) — the technology of the moment — declined?
Wanting to dissect these areas further, Information Age spoke to Abhas Ricky, managing director, global head — strategy and transformation at Cloudera.
The data journey: It’s only the beginning for digital transformation — Big Data LDN
Data visualisation
Data visualisation is important, because it is a graphic representation of the data — it helps the data leaders tell a story and convey that message to other stakeholders within a business.
But, “if leaders do not have an understanding of what the business outcomes are, at a micro level, attached to specific investments or projects in the data landscape, it becomes very difficult to tell that story,” explained Ricky. “And data visualisation is a key aspect of that, because it is a method of storytelling. It is a way of presenting the larger vision, backed by metrics and facts to generate insights.”
Despite this, data visualisation is holding up data leaders achieving their business goals. The data representations are not being conveyed convincingly and decision-makers are demanding new standards for how they interact with their business data.
According to Ricky, “the fundamental problem today is a.) people don’t have an understanding of ROI, b.) even if you do, in the majority of the cases you don’t have an understanding of the ROI that you’re impacting or bringing in, because everyone latches onto the larger business problem. But actually, you’re only addressing a very small part of it through a specific project, with a specific tech or specific vendor.”
Rohan Kumar, corporate vice president, Azure Data at Microsoft, also offers some insight into data visualisation challenge.
He said: “The challenge with visualisation is that only so many users can visualise the datasets that have come from your ERP or CRM systems, and IoT devices in your data warehouse. Resource typically runs out in a single working cluster.”
Microsoft president, Brad Smith, recently stated, “We will begin this next decade with 25 times as much digital data on this planet as we had in the year 2010”. This is only good for organisations with the skills to gain actionable insights. For those without the skills, data can be a hindrance and can quickly prevent organisations from realising genuine business benefits.
Overcoming the data visualisation barrier
Changing an organisation’s structure was identified as one-way data leaders could overcome the data visualisation barrier.
Ricky said: “Data visualisation is a way to test hypotheses. The visualisations are built by technically savvy people, however you need people with a domain expertise to iteratively generate the hypotheses to be tested. Many organizations are still struggling with data silos, preventing this rapid iteration and hypothesis testing that ultimately delivers insights.”
It’s all about understanding, breaking down organisational silos and bridging the gap between business and IT.
The key to harnessing unstructured data? Visualisation
The second way to overcome the data visualisation barrier, according to Ricky, involves business value. When allocating budgets for business projects, everything has to be tied to business value. “These are mutually exclusive and have to be collectively exhaustive,” he explained. For a project — such as gaining a single view of the customer — organisations will need applications, infrastructure and compute — “each one of those vendors will attach value to those individual use cases. This is double or triple counting, which you don’t want,” continued Ricky.
Instead, organisations should create a system where a specific business value is attached to a specific technology — that will help simplify the process and bridge the gap between business and IT, which will create a better understanding of the ROI.
Unlocking data in enterprise systems: a Network Rail case study
A decline in AI and machine learning
Another of the report’s more surprising findings was that the usage of AI and machine learning by UK organisations had decreased 21% in the last year. Given that AI and ML are the ‘technologies of the moment’, why has this happened? Perhaps the hype around these zeitgeisty technologies is dwindling or end users lost faith in the promises of vendors?
However, instead of the use of AI declining, Ricky suspects that certain actions have just been reclassified.
He explained that AI has used as a big umbrella word; “a lot of organisations were even classifying simple data engineering projects under AI, because it was easy to get budget sanction for it — it was termed as futuristic and not legacy [keeping the lights on].”
As the AI hype cycle has started to mature, many of these projects that were once classified as AI projects have now been recategorised into data engineering, data governance or data security, for example.
A disconnect between c-suite and mid-management
Ricky said he believed there was a fundamental disconnect between the c-suite and middle management, regarding the applications of artificial intelligence. “The majority of CEOs,” he said, “will describe a future vision of how AI is transforming a specific industry or ecosystem.
“But this is not in the mainstream of AI production use cases. If you speak to middle management, they are still struggling with data engineering, data management, data cleaning, data governance and data security. These ‘basics’ are fundamental in delivering AI.
