Modern businesses are increasingly forced to halt operations while their business intelligence professionals run ever more complex data loading and analysis. The data economy cannot be sustained if the information that it depends on continues to be tied up in processing for hours, days or even weeks.

The incumbent giants of data warehousing such as Oracle and cloud-ported alternatives such as Hadoop have caused a rift between the data-hungry users of today and the infrastructure of yesteryear.

These legacy solutions under deliver on their performance promises and effectively charge clients to hold their data hostage, whereas a true cloud-based solution would be more capable of flexing to the needs of its users, charging for precisely what’s used.

>See also: Legacy to cloud: former founder and CEO challenges old business

This article addresses the complacency plaguing the industry and explores the means by which we might end the data struggle.
__________________________________________________________________________________

Data is the driving force of today’s economy. There’s no escaping the endless reference to storage and analytics in the media, yet suspiciously absent in these discussions is any mention of the problems faced by users. In this supposed age of innovation, complacency is king in data warehousing.

According to a recent study in the U.S. by Dimensional Research, 90% of data professionals dubbed analytics as ‘very important’ to their organisation, yet 92% have faced issues with flexibility of data – either being forced to over-provision for their needs or face a shortage during peak activity. It’s time to address the usability issues corrupting analytics, and adopt a true cloud service free of bottlenecks that caters on-demand to the needs of your business.

The key downfalls of traditional data warehousing are that it has not kept up with demands for flexibility or exponential growth, and user concurrency is simply not accounted for.

Large datasets can cripple a system when approached with ad-hoc queries: the process freezes it while data is hashed, delaying the work of other users and placing artificial restrictions on business processes.

In a similar manner, semi-structured data will incur a time penalty as it is ‘flattened’ by data admins, and any attempt at concurrent queries will deliver peculiar idiosyncrasies depending on how each was run.

>See also: Preparing your business for a digital transformation

By separating storage, compute and services into separate layers, a cloud architecture could handle multiple transactions against the same data concurrently.

Legacy services cannot sustain the demands of modern enterprise: data is effectively vaulted up behind time-locked doors. In real terms this is reflected in the astounding 88% of organisations who have experienced failed data initiatives.

In 2016, 16 zettabytes (ZB) worth of data was produced globally, according to a recent study from IDC and Seagate. By 2025, this is expected to reach a whopping 163ZB, largely driven by the emergence of cognitive systems and the rapid growth of the IoT.

In today’s business landscape, it’s common for companies to aggregate hundreds of Terabytes each month. A traditional warehouse solution would see this scattered across thousands of data silos, following no logical distribution.

What’s more, it’s virtually impossible to move and will demand resources long after its initial purpose has passed. From an analytics perspective, this architecture is downright unsustainable.

Here the advantage of cloud warehousing lies in the unmatched flexibility of on-the-fly resource-allocation, to combat stagnant data hoarding.

>See also: Propelling legacy systems into real time

Drawing on the infinite resources of cloud architecture, scalability is a simple matter, resolved in minutes. By paying only for what you need, your budget can account for periods of heightened activity, and you will ensure data is kept up-to-date as a matter of habit.

The data economy cannot be sustained if the information that it depends on continues to be tied up in processing for hours, days or even weeks – even then the results frequently fail to show a unified order.

While many businesses took a leap of faith with newer ‘cloud-washed’ services they have no doubt felt their confidence taken advantage of. These have proven themselves mere cosmetic alterations – ported versions of the same legacy architectures – equating to little more than the frosted-tips of a pensioner lamenting his feeble grasp on what ‘it’ is.

‘It’ is on-the-fly capacity planning, semi-structured data handling, uninhibited data transformation – a unified data environment where everyone works from a single version of the truth.

A true unconstrained data warehouse must be built for the cloud, not merely ported to it. No modern business can afford to wait a week for a number-crunch. What we need is a solution capable of tackling all the datasets, all the time – ideally in under an hour.

>See also: Robotics process automation and the robotised enterprise

While analytics have been driving technologies and marketing for decades, there is an illusion of youth which disguises reality: data warehouses are not so spry as one might think. Rather these services, instrumental in building the technological world as we know it, are rusty at the hinges. They creak along through queries at a rate that cripples the businesses they serve.

You are not asking too much when you call for easy access to your data, or for multiple queries without performance penalties. Conventional data warehouses have refused to innovate and it has taken too long for customers to realise the effect this is having on their performance.

The limits of infrastructure are an intrinsic issue with on-premises data centres. In the cloud, there are no limits: data warehouses should be scalable, omniscient and portable, not clad in iron at the bottom of a dark data dungeon.

Sourced by Thibaut Ceyrolle, VP EMEA at Snowflake Computing

Nominations are now open for the Tech Leaders Awards 2017, the UK’s flagship celebration of the business, IT and digital leaders driving disruptive innovation and demonstrating value from the application of technology in businesses and organisations. Nominating is free and simply: just click here to enter. Good luck!