People often associate the term ‘web-­scale IT’ with the titans of the internet age like Google, Amazon, Twitter and Facebook. These companies were forced to engineer bespoke technologies from the ground up to accommodate vast databases and customers in their hundreds of millions, and take big gambles and investments to match – something which is beyond the reach of your average conservative enterprise in a more traditional industry.

But this is not the case – these days companies just a fraction the size of Amazon or Google have to balance the budgetary pressures of needing to do more with less, while driving business growth, and so are embracing the philosophy of web-scale IT.

In March last year IT analyst Gartner said that by 2017 web-scale IT will be an architectural approach found in as many as half of global enterprises across all sectors, arguing that if enterprises want to remain competitive in the digital world, leveraging these kinds of economies of scale by emulating the business mindsets of the internet giants will be a must.

> See also: Are you ready for petabytes-scale storage?

This is because the word ‘scale’ is misleading – the web-scale IT approach is not about size. The Amazons and Facebooks of the world got to be where they are not because they invested big to begin with, but because they designed their architectures, their processes and their practises to handle the relentless pace of technological change, able to cost-effectively grow their infrastructure incrementally as they needed to, and that is the real key: agility.

As Howard Ting, VP of marketing at storage and virtualisation firm Nutanix explains, there isn’t one set definition or way to do web-scale IT, but there are a few core principles to take onboard.

‘As apps continue to change from big monolithic systems like Oracle and SAP to more micro-service, web-oriented applications, the infrastructure is also under tremendous disruption,’ says Ting. ‘So we’ve seen a big shift in what companies are looking to deploy in a web scale IT organisation, and it looks nothing like traditional IT infrastructure.’

This new web-scale IT approach comprises three fundamental elements: a change in infrastructure, a change in IT to support agility, and a fundamental cultural change.

Little and often

In a traditional IT environment, enterprises run one large, monolithic storage array, much like a mainframe or supercomputer.

‘Enterprise IT has been conditioned to buy big and buy ahead,’ says Ting. ‘Over time, scaling storage becomes very challenging because you don’t know how much storage you will need for the business as it grows, and you can’t simply add more.’

‘Companies have traditionally provisioned storage three or four years ahead of what eventual capacity will be, and as a result they tend to oversize and over-provision and get very poor returns.’

However, Companies like Amazon build what’s known as distributed architectures. In a distributed system, companies build machines that are very small and then cluster them together to form a big pool of resources. When they need to scale, they simply add one more server.

‘Web companies operate at a very high degree of volatility,’ explains Ting. ‘Google has become the dominant search engine, but there are several that came before it and nobody really knew which one would become the dominant provider. So all of these providers had to start smaller and get to a point where the could scale infinitely without diminishing returns and with predictable economics.’

The main benefit of web-scale infrastructure is that it’s not a ‘rip and replace’ architecture – legacy deployment can coexist with a web-scale architecture, and if your existing IT infrastructure no longer meets the needs of the business from both a commercial and technical perspective, you can stop investing and migrate some applications, expanding to meet your business needs as you go.

As Ting explains, this approach is disrupting the whole storage market, forcing vendors to take a u-turn on long-held sales strategies.

’One big change I would advocate for is a ‘buy what you need’ strategy,’ says Ting – something which we are already starting to get conditioned for by the public cloud. ‘I think one of the most important things businesses should start to reevaluate when looking at taking up web-scale IT is the consumption of infrastructure. With SAS, Salesforce and other IT as a service platforms like AWS you only pay for what you use that month. I would say challenge the vendors: say I only want to buy what I need right now and be able to scale it, how can you help me do that?’

Agility and Software

The second defining element in web-scale IT is the ability of IT processes to drive agility, and this means software is a key driver of value.

‘If you look inside these web companies, they’re all based on commodity hardware – just rows and rows of x86 servers,’ says Ting. ‘The real value is through software.’

‘In web-scale IT, the efficiency and ratio of people to machines in a data centre is able to be fundamentally different thanks to powerful software,’ says Ting. ‘You can now have one administrator managing 500 machines. In the case of Facebook you can have one engineer managing 20,000 machines. That's orders of magnitude more efficiency.’

> See also: Bringing web-scale IT to the corporate data centre

In a software-defined approach, software does many of the mundane jobs humans have traditionally done in data centres, such as patching, updating software on machines and maintenance. Automation means a higher degree of efficiency, and this means having the ability to scale up service capacity based on demand patterns and rapidly scale capacity in peak business periods.

So with all these benefits to businesses, what is holding them back from realising the potential of web-scale IT? As Ting explains, it’s more than just a technology arms race.

‘One of the key inhibitors we experience when we go into a new organisation evaluating our technology is that you get all the support from the change agents, the architects and the c-level folks that see the business needs to transform, but then encounter resistance from dedicated teams, those that have managed old infrastructure,’ says Ting.

‘Maybe their skills have become less valuable, they have had to become generalists, and there's resistance – it's just human nature, but eventually it has to be dissolved, because if companies continue to resist and will live in the past they’re going to become disadvantaged and that will show in their business results. This is a fundamental change in not only the type of technology we deploy, but a change in organisational boundaries, people and culture, so it’s a painful change for many.’

Culturally, web-scale IT relies on a basic change in attitude, and one that many more conservative companies might balk at.

‘A traditional enterprise might make one big change in 100 days,’ says Ting, ‘and if said you were going to make 100 changes in one day they’d stop you and become very risk-averse, and freak out because they’re not accustomed to that. It’s a fundamentally different speed of operating. But go inside a company like Google and say ‘it’s going to take us 100 days to roll that out, and you’ll get laughed out of the room.’

Web-scale companies have tremendous advantage over everyone else in terms of how quickly they deliver products and services – and that’s an undeniable competitive advantage.

‘They are able to have a much much higher degree of scalability, resiliency and efficiency than traditional enterprises are accustomed to,’ says Ting, ‘But first they have to be prepared to step out of their comfort zone and take risks.’