Healthcare scientists examine vast volumes of genome sequencing data to crack some of society’s most difficult diseases. Retailers are using data analytics to predict which clothing range will fly off the rails next season. Mobile phone companies are looking at millions of call detail reports to second-guess customers’ needs and preferences. Police forces are following patterns of citizen behaviour to plot the next crime hotspot. This is big data in action.
Big data is like cloud computing in that the term has moved out of the IT function into the mainstream. Consumer news reports are regularly filled with stories of how the big data phenomenon is transforming the way organisations of all types identify insight, patterns and trends.
>See also: Is big data dead? The rise of smart data
So what do we mean by big data? It is essentially datasets either too large or complex for traditional data processing applications to cope with. The ‘three Vs’ are often also used as a basis for defining big data: volume, variety and velocity. Regardless of the scale of data that is currently considered big data, the one constant is that it will grow by orders of magnitude over the next five years as biological structures, interactions, transactions and the Internet of Things create more and more data.
Big data is making waves in almost every corner of the commercial and public sector. A global study conducted by Vanson Bourne, commissioned by Automic, highlighted that 60% of companies have either implemented a big data strategy or are in the early stages of implementation. The figure rose to 66% among energy and utility companies, and 63% among financial services organisations.
There are some sharp contrasts between countries though. German is leading the march to big data, with 73% of organisations either implementing a big data strategy or in the early stages with a clear plan. This compares with 72% in the US, 56% in France and 41% in the UK.
Laggard countries like the UK would do well to catch up with their German counterparts, as a big data strategy offers potential competitive advantage. As companies store more transactional data in digital form, a data-driven strategy enables them to make more agile decisions based on more accurate and detailed performance information.
Big data allows ever-narrower segmentation of customers and therefore more precisely tailored products or services. The sophisticated analytics enabled by big data can also be used to innovate the next generation of products and services.
There’s a problem though. The research also revealed that 29% of respondents know they need to make use of big data – but are unsure about how to implement it. Of the respondents whose company had a big data strategy underway or was in the early stages of developing one, only 24% thought their planned big data strategy will be or has been completely successful. This figure dropped to just 12% in the UK.
What is holding them back from being successful? The research indicated that 96% of organisations are experiencing barriers to innovation within their company. In the UK, the biggest barrier was cited as budget constraints (54%), contributing to a slower pace of innovation.
>See also: Intelligent business demands a bold future for big data
It’s clear that big data is complex, but it’s also rich with potential. Organisations that overcome their big data challenges can discover data correlations and patterns that before would have remained hidden. This means they now have access to more accurate information that can influence their business.
By automating the processing of large amounts of data and the loading of big data into in-memory databases, businesses can reach the predictive analytics phase more quickly – and speed up crucial decision-making.
While automation helped speed up data analysis before the arrival of big data, using manual data processing would now be laughable in this instance. It translates data into a coherent format that the business can understand, and also helps businesses prepare for the Internet of Things.
Sourced from Vladi Shlesman, Automic
