The global shortage of data scientists is affecting every large business. But don’t worry – the ability to analyse data is something you already have inhouse

Globally, we are in the midst of the largest shortage of data professionals in history. Combined with widescale disruption, an inability to hire at speed, and a rapidly changing competitive landscape, the impact of these shortages – from expert data scientists to in-department knowledge workers – is keenly felt.

For a specialism that is only 20 years old, this understandable gap between what’s available and what’s required is explainable. In reality, the most staggering data-driven insights of the last century have been delivered from subject matter experts – not data scientists – working with their own unique expertise and limited datasets.

Florence Nightingale, for example, revolutionised and reformed the medical field with health statistics and mathematics – linking hygiene to improved medical outcomes. Many businesses today conflate the need for data scientists with the need for data insights.

‘The ability to ‘speak data’ needs to become a check box on any hiring manager’s brief when searching out new hires’

Classrooms across the world staple phrases like “measurable goals are achievable goals” or colourful collages extolling the virtues of SMART goal setting strategy. But what happens when the things that are “known” and measured are – in and of themselves – wrong? What happens when focusing on measurable factors results in incorrect outcomes?

You don’t need more data scientists

Businesses need people with the business context of questions being asked and data skills to turn information into insights. Gartner highlights that, by 2023, data literacy will become an explicit and necessary driver of business value – linking successful business outcomes to the ability of workers to “speak data”. The challenge, therefore, centres around not only the need for fast insights, but also with hiring the right people to deliver those fast insights.

There are two core barriers to meeting this challenge head on. The first is the well-publicised understanding gap – the misconception that hiring more data scientists is the solution to delivering these insights. The second is the mindset that layering on technology after technology can make up for a shortfall of human ingenuity.

Instead, the solution lies with the experts already working in a business – in-department knowledge workers with the context of the problem they are looking to solve, the wider context behind the question being asked, and the implications of any proposed. These workers may not immediately have measurable “data science” skillsets, but fully utilising these existing workers – and widening commonly accepted skillset definitions for new knowledge workers – is one of the single most effective productivity drivers that businesses can integrate today.

Upskilling your own knowledge workers

In developing this new data strategy, highlighting knowledge workers as the missing link between data and data scientist is essential. Bringing these in-department experts – and the unique knowledge they hold – to the forefront of decision-making sits as the second strategy for delivering the data skills necessary to businesses today. Businesses need to look at what resources are held in-departments, and how they can be utilised.

Case study – a logistics company

In a logistics company, for example, if you were looking for the most optimal route for a delivery, you would ask the driver – not the data scientist. While the data scientist can factor in macro trends, the driver is able to assess the micro changes that may otherwise be invisible. In manufacturing, it’s the equivalent of a longstanding worker listening to a machine and knowing something sounds wrong.  

When integrating strategies to mitigate the data skills shortage conundrum, the solution is fully utilising the value of hard-won direct experience – putting tools in the hands of the people who can deliver the most value. For existing workers, this is achievable through upskilling programmes – enabling these knowledge workers to solve their own challenges with data.

Through a hiring lens, this requires a refocus and a shift in business understanding. Every department is able to use – and invariably does use – computers. Just because it’s electronic doesn’t mean it’s the sole responsibility of the IT team. It’s a whole-business resource.

The same mentality shift needs to happen with the generation of business insights. The challenges posed by the sheer volume of data created means that it’s impossible to sequester the responsibility for data insights exclusively within data science teams. Businesses need to reassess the skillsets they look for across all teams beyond a linear approach to their specific remit. Instead, the ability to “speak data” needs to become a check box on any hiring manager’s brief when searching out new hires.

This is what we mean when we refer to data democratisation and upskilling. While not every worker needs to become a data scientist, there are hidden insights locked away in – for example – departmental legacy spreadsheets which are entirely invisible to formal data teams without a democratised responsibility for data work. This whole-business approach to data-driven insights is vital.

Cultivating inhouse expertise

On average, data workers leverage more than six data sources, 40 million rows of data and seven different outputs along their analytic journey. In practice, this means that data scientists simply don’t have the time to solve every problem they need to. Instead, these experts should be leading on data strategies while supported and buoyed by a raft of in-department experts – workers who can provide valuable context and unique perspectives to colour strategic decisions.  

The USP of the five most profitable companies in the world isn’t cash reserves or real estate – it’s the ability to collect, analyse and act on the data they hold – at scale – to refine the value and make effective decisions. Moreover, these analytically mature organisations have developed an organisational structure which not only allows but facilitates the communication of this data. Delivering this new cultural structure – for both internal training and future hiring – can be achieved with the right strategy.

Three ways to help staff become their own data scientists

• Build awareness and excitement: people need to understand what the benefit is for them personally, and they need to be excited and motivated to make that change
• Deliver Enablement and training: People need the necessary skills to act on that initial impetus
• Support and sustain success: with an understanding of the value on the table, the priority then pivots to supporting and encouraging desired actions to deliver successful change

Gartner research estimates that – in 2019 – we exceeded one billion knowledge workers globally. These knowledge workers are your in-department experts. These knowledge workers not only exist but are already employed. They are the people who are paid to think, who use information to inform those thoughts. They are also critically underutilised.

Today, the data skills shortage presents – at all levels – as a critical need for data workers with the skills, data literacy, and accessible tools to operate effectively in an increasingly data-driven environment.

Further, that shortage also highlights the impact of lacking diversity and inclusion within the data analytics space itself… a lack of diverse viewpoints ultimately leads to a narrow understanding of both the challenge and the solution. Just as it would be ineffective to have a master mechanic perform an oil change, so too is it ineffective to have a data analytics team exclusively – or even heavily – made up of data scientists.

Build your way out of the data science skills shortage

Ultimately, the best strategy to build your way out of the data science skills shortage and bridge the gap is simple: work to utilise, upskill, and enable the experts already on payroll to support your existing data science teams – enabling them to deliver the right insights, at the right time, for business benefit. With that benchmark in place, then refocus and reenergise hiring strategies with an overarching focus on the priorities most important to your business.   

Libby Duane Adams is co-founder and chief advocacy officer at AI analytics company Alteryx

Digital skills gap costing UK economy £12.8bn – Virgin Media O2 research has revealed that a shortage of digital skills is costing the UK economy £12.8bn, as a cost-of-living crisis continues across the country

The value of data — a new structural challenge for data scientists – Felipe Henao Brand, senior product manager at Talend, discusses how to identify the true value of data — a new structural challenge for data scientists

How to empower your data scientists in the era of edge computing and AI – Dan Warner, CEO and co-founder of LGN, discusses how data scientists can be empowered in the era of edge computing and AI

You can find your next career move over at the Information Age Job Board