The computing world has been shaken by a groundswell of companies moving to the cloud to solve their data storage challenges. Established hardware vendors, such as IBM, Dell, Oracle, Cisco and EMC, have been disrupted by cloud services from Amazon, Google and Microsoft amongst others. More than two thirds of enterprises are now using databases in the cloud.

Despite dominating the industry in the 1990s, Dell and EMC have now merged in the biggest acquisition in tech history – which to many is hard evidence that traditional tech companies can no longer survive on their own.

The cloud has set the computing world alight much faster than expected, and the deal between Dell and EMC is a bid for survival in a shrinking market.

>See also: Big data in the cloud – where next?

The move to cloud makes sense on both a financial and logistical level. The cloud is far cheaper than traditional storage options and can be scaled affordably thanks to converged infrastructure and shared services.

From a practical point of view, the cloud allows businesses to store huge amounts of data, using minimal hardware. As a result, nine out of ten organisations are currently using at least one cloud application, and business spending on hardware is expected to drop by 13.6% in 2017 as more companies migrate to the cloud.

However, the transition is far from seamless – there are several serious logistical issues associated with a move to the cloud. Amongst cloud vendors a battle brews over how to make the leap from firewall to cloud.

For really large volumes, there are solutions that involve uploading the data onto a device that can then be physically uploaded to a remote data centre. For non-transactional data (data that doesn’t change very much), users can simply take a snapshot and copy the data. 

This approach works for non-transactional data, particularly because multi-terabyte transfers can be so expensive, but how do organisations deal with transactional data that changes all of the time?  By the time an organisation has copied the data, a large portion of it may have already changed, meaning the data it has copied is stale.

What happens when businesses need access to their information in real time, when a 24-hour delay will have major repercussions on strategy and operations? In today’s world, where round-the-clock access to data is becoming increasingly necessary, we need to come up with a better solution.

In 2006, Jeff Bezos approved an ‘experiment’ to create a business using Amazon's retail computing infrastructure that was incredibly efficient. In its first year, 180,000 software engineers signed up – primarily for an easy way to test the apps they were building.

Those acorns soon grew into huge oak trees with test apps including Netflix, Airbnb and Tinder starting to use Amazon Web Services (AWS) rather than their own servers in their own data centres. AWS is now a $7 billion business and Gartner estimates that it will become a $50 billion business within the next four years.

So how is this affecting the broader architecture of the data warehouse?

The current state of play within the data industry can be described in two words: consolidation and innovation. The early adopters – AWS, Google Cloud and Microsoft Azure – are forcing established vendors to change their business models, whether through partnerships with younger firms, or through internal development.

>See also: The great IT myth: is cloud really less secure than on-premise?

These partnerships will come to define the cloud services industry, with established vendors bringing tried and tested reputation and industry-clout to the table.

By spending time and money developing the process of cloud migration, businesses can begin to formulate a system of data transfer that is painless, seamless and occurs in real time.

Ultimately, the expertise and experience that established vendors bring to the table will submerge it for good within the mainstream. Until the value of combining old and new is recognised, the cloud migration process will remain difficult. Key to the solution is ensuring that the data can be moved in and out of the cloud in an easy yet secure manner.

David Richards, co-founder and CEO, WANdisco