The big data and artificial intelligence boom is expected to create 56,000 data science positions each year from now until at least 2020 in the UK alone. However, the number of fully skilled professionals in this area doesn’t appear to be growing proportionately to the number of positions available – in other words, there’s significant mismatch.

In addition to this, there is a widespread misconception about the ‘modern’ data scientist, including what they do and how they do it. The term summons up images of a mysterious cabal of white coat-wearing people, beavering away on arcane topics of only theoretical interest to their employers. The reality is quite different – both in terms of the role and its practical importance to the organisation.

>See also: Machine learning and AI is changing how data science is leveraged

It’s also why businesses should all be deeply concerned about the data scientist shortage, which could soon be costing billions in lost opportunities across a range of industries.

A data scientist ‘day in the life’

Money may make the world go round, but today it’s data that greases the wheels. Each day, people and organisations create around 2.5 exabytes of information, in the form of structured and unstructured data.

This information – thanks to its size, its format, and its dispersal among so many different platforms and silos – is a wasted asset without data scientists who can interrogate the raw data into insights that can be applied to solve real-world problems.

A good data scientist is more than a mathematician, statistician, or writer of algorithms – although these skills are obviously central to the role. Data science is more than just number crunching: it is the application of various skills to solve particular problems in an industry.

>See also: How to choose a data science vendor

This means you can’t drop a statistics graduate straight into a data science position and expect them to start delivering insight from day one. The job requires far more than theory – data scientists need to have a thorough understanding of the domains in which their insights will be applied. So on top of maths, data engineering and visualisation, a data scientist might also need a high level knowledge of supply chain, finance, logistics, human resources, or any other line of business.

Little wonder, then, that data scientists can earn so much – and why they are in such short supply.

Making the investment

Good data scientists do not come cheap. Recent research by Hired found that the average salary is £56,000 in the UK and $129,000 in the United States, with salaries significantly higher for more experienced experts.

But, used wisely, data science can represent one of the best bargains a business can make. By turning billions of bytes into actionable insight, data scientists can solve long-standing business problems, identify inefficient processes, develop new revenue streams or markets, improve data security, enhance customer service, develop tailored services – and provide answers to all the unknowns that a modern organisation faces.

>See also: Data scientists are the key to solving real world problems

Data science might once have been a luxury, but that’s no longer true. Such is the business edge that the discipline confers on its users – such as speeding time-to-market, to take just one example – that organisations in practically every industry need data science to remain competitive. But getting their hands on the right talent is becoming increasingly difficult.

More skills to pay the bills

The scale of the data science skills shortfall is so large, you’d almost have to be a data scientist to make sense of it. The European Commission estimates the EU will need an additional 346,000 data scientists by 2020, while in the US estimates vary between 100,000 – 190,000.

To put that in perspective, around 8,000 people graduate with a data science or analytics degree in the US each year. That’s great news if you’re one of the lucky few – less so for businesses that urgently need talent to unlock the power of data residing in your silos.

>See also: Data 50 Awards 2017: winners revealed

Even then, the low number of data science graduates is only part of the problem. As discussed, no-one comes out of university with all the skills they need to make a difference in the corporate environment. No matter how many technical skills they have, graduates need several years at the coalface where they can learn to apply theory to the specific business challenges that they have been tasked to solve.

The challenge facing businesses is twofold: how to access the raw talent in the first place, and then how to commoditise these skills so that they can be applied to different areas of the business. In Silicon Valley, it’s quite common to see businesses take the cream of that year’s crop of graduates and put them to work in the research and development department, where it is hoped they will add some value while they learn enough about the business to be useful elsewhere in the organisation.

Another approach to mitigate this problem is to have the talent from university to be groomed industry ready. This is possible if there is a constant and water tight engagement between the academia and industry, and possibly Government funding agencies.

For instance, a part of the curriculum that graduates undergo could be dedicated to gaining industry knowhow through paid internships and a dissertation on the practical problem addressed by the would-be data scientist in the company that s/he selected. While this will add a significant value to the “day one worthiness” of these graduates, industry also stands to gain by infusing fresh thinking from academia to solve data science problems.

>See also: The 4 secrets successful CDOs will know about data

The skills gap is an ongoing issue facing businesses everywhere. Whether this is the right approach will take time to ascertain. In the shorter term, organisations need to develop a strategy for how they acquire the talent they need, including offering the right incentives to attract the best data scientists and on-the-job training to ensure that they can start delivering value as quickly as possible.

This may seem like a considerable investment right now – and perhaps one that businesses don’t feel comfortable in making – however; it should be viewed as a strategic, and an incredibly important step that is guaranteed to pay dividends in the future.

Sourced by Dr. N. R. Srinivasa Raghavan, head of the Data Sciences Team, Infosys