The mass digitisation of content and the unstoppable drive towards retaining as much data as possible has created exponential growth in the amount of unstructured data that businesses are managing.
These challenges have tested the limitations of traditional storage infrastructures – backups & restores are taking longer, migrations to new storage systems are labour intensive, and scaling storage using these technologies is becoming more expensive and time consuming.
Object Storage has established itself in the market in response to these challenges, offering what refers to as, ‘a promising future for organisations trying to balance scale, complexity and costs’.
Almost all of us are already Object Storage users in one way or another, whether it’s working on Google Docs, sharing files via Dropbox, pictures with Instagram or checking Twitter.
How does Object Storage work?
Objects are data, just like files. But unlike files, they are not organised hierarchical but exist on the same flat storage pool as any other piece of data. Objects exist all on the same level and cannot exist within another object. Just as files, objects have metadata.
> See also: Five years open: the trials and tribulations of OpenStack
Objects have extended Metadata though as they are assigned a unique identifier that allows the server or user to find the data without knowing the physical location of it. This approach makes it easy to automate of streamline data storage in cloud computing environments.
Imagine going to the Opera. You check in your coat at the coatroom and get a tiny receipt with a number on it. You do not know exactly where your coat is or even if it is moved several times during the opera, but you don’t need to as the cloakroom attendant knows where it is.
In this analogy the object storage’s unique identifier represents your receipt, your coat is the data and you are the user that wants to have the coat back at some point, only knowing the receipt number.
This practical concept, that has worked for hundreds of years, is the basis for Object Storage.
Market growth
This exponential hyper growth in data is consuming vast amounts of storage space – IDC estimates that a mind boggling 1.8 zettabytes of digital data will exist by 2020 (to give it some context, a zettabyte is 2 to the 70th power bytes, or around a billion terabytes).
Unsurprisingly, cloud service providers are among the organisations who have become enthusiastic adopters of Object Storage to deliver scale-out solutions which are highly resilient to failure, and which grow ahead of demand – in effect, they need technology which is ‘forever live’.
Beyond the familiar tech brands, Object Storage is also used across the enterprise space for on premises storage, supporting applications that create and utilise massive amounts of unstructured data, including content media storage, backup and archiving, data analytics, private cloud and file distribution and sharing.
S3 – a de facto standard for Object Storage
The fundamental reason why increasing numbers of us are Object Storage users is because that so many organisations now rely on Amazon S3 to deliver part or all of their cloud storage infrastructure.
As the Object Storage service within the AWS portfolio, S3 has played a central role in the broader AWS success story – described recently as ‘the greatest disruptive force in the entire enterprise technology market today’.
The link between public and private Object Storage solutions is growing as more businesses opt for a hybrid approach to their storage needs. The dominance of S3 in the public cloud storage space means there are great advantages to using it for on premise solutions as well.
For any organisations that use Object Storage as part of a hybrid solution – and there are many – S3 has quickly become a generally accepted requirement for delivering affordable, high performance and scalable data storage across their entire storage strategy.
So in effect, S3 is now operating as the de facto standard for Object Storage technology. Other approaches have been brought to market, such as CDMI, which was defined and created by the Storage Industry Networking Association (SNIA), but has seen limited adoption by Object Based Storage (OBS) vendors, and has been ignored by public cloud providers.
> See also: Are you ready for petabytes-scale storage? Here’s what you need to know
Another is OpenStack Swift, which is a fully open source API standard, and has built a following – it could be described as the 2nd most popular cloud storage API, and most OBS vendors are adding support for Swift to make their storage Plug ‘n Play with OpenStack Nova.
But its impact is a long way short of the outright domination enjoyed by Amazon S3, despite the fact that it wasn’t created as an attempt to standardise anything. Primarily driven by sheer popularity and weight of numbers, S3 has the largest number of applications written to the specification – today, more than 4,000 ISVs support S3.
With over 80% of future storage capacity expected to be required for unstructured data, the momentum behind Object Storage and S3 is only likely to increase. AWS is the fastest growing enterprise technology company in history, and six times larger than its nearest competitor.
Revenues and user numbers are accelerating rapidly – one valuation puts the business at $160bn, which is on a par with IBM.
The central role of S3 in this success story and the development of enterprise storage strategies will see its role develop to a ‘must have’ as organisations seek to balance the need to provision for ever increasing capacity, while keeping a lid on cost.
Sourced from Neil Stobart, EMEA Technical Director,Cloudian
