More and more organisations are realising the benefits of moving away from legacy technology systems to modernise, automate and innovate. However, unless they have a robust data strategy in place, it’s impossible for them to realise the benefits that AI can bring to their business.

The data challenge

At the heart of any data strategy should be the aim to reach a single point of truth. But the sheer amount of data that most companies need to process, together with the various forms that data can take, is often a major barrier to achieving that vision.

Today’s businesses must process more data than ever – and this volume will only continue to increase over the years ahead. This data is likely to be captured in various forms – from structured databases to unstructured data sources like word documents or emails – and stored in several different places.

Take, for example, a large company with seven different customer-facing locations, each of which has its own master sets of data. As each set has been captured independently by location, there’s no simple way to report on what an individual customer is buying across those seven locations. This can have all kinds of implications, from a reduced ability to drive discounts centrally or pre-order items that are selling well to avoid stock shortages.

What’s more, storing data on disparate systems not only means a company is potentially missing out on valuable business insights, but the use of ‘dark’ data means they could face rising storage costs while increasing security and compliance risks.

Driving a successful digital transformation strategy

Gordon Cullum, CTO at Mastek looks at what it takes to drive a successful and evolving digital transformation strategy. Read here

Cleansing and centralising

For optimal data intelligence, data needs to be cleansed and centralised. However, I estimate that, on average, businesses don’t know about 80% of the data they have; where it is or what it is. So, the first task when formulating a data strategy is to find it all.

Next, it’s important to identify and discard anything that’s no longer valuable. For example, if a retail company finds an order made by a customer 30 years ago who hasn’t bought anything since, it’s safe to say they can delete that.

Once data has been cleansed, it’s ready to be stored and processed in a centralised data lake, making it easier to manage and analyse.

Here are a few points organisations should consider when storing significant amounts of data in one place:

1. Security
A robust plan for backup and recovery is fundamental. Look for systems that use solid data protection policies such as WORM (write once, read many).

2. Scalability
For maximum flexibility, choose a platform that will scale according to the demands of your business. Ideally, this should be in terms of performance and capacity.

3. Format
Consider which format you need your data to be available in to ensure you get the most out of it. For example, this could be a Hadoop File System or an object storage instance.

4. Translation tools
While a single data lake is the Holy Grail, until every app designer in the world decides to follow a uniform way of doing things, this is virtually impossible to achieve. However, using a translation mechanism such as Kafka makes it possible to manage data across different apps allowing businesses to build a data lake.

5. Location
When it comes to choosing a storage location, my recommendation is to opt for a cloud-based service to start with, as on-site data centres are expensive.

The data journey: It’s only the beginning for digital transformation Big Data LDN

In one of the opening keynotes from Big Data LDN, Doug Cutting — Chief Architect at Cloudera — discussed the importance of the data journey in the pursuit of digital transformation. Read here

When an organisation has centralised its data, it’s time to look at analytics. What are the business problems you want to solve? What keeps you up at night? What are the insights that will make the most difference to your business?

A clear set of objectives is crucial and, once they’ve been agreed, a test run with a smaller data set will reveal whether a business is asking the right things and using the right algorithms to answer its burning questions.

Each of these steps contributes to a successful data strategy; which should ultimately lead to a single point of truth. With such a strategy in place, the business is ready to embark on digital transformation and take advantage of the business opportunities that AI-driven data analytics can deliver, including improved decision making and customer service, plus increased productivity, to deliver a real competitive advantage.