A fundamental change to how networks are designed and run is underway. The monolithic routers and switches that have for so long dominated carrier and enterprise networks and form the backbone of the internet are gradually being replaced by smaller ‘white boxes’ based on off-the-shelf ‘merchant’ silicon and software-defined networking (SDN) technology.

So far, the revolution has taken place largely behind closed doors. But with about half a million white box switches being installed each year, if you were to visit any of the top ten data centres worldwide, you would find row upon row of white boxes bearing the logos of lesser-known vendors from China and Taiwan.

To give you an idea of the significance of this change, consider that arguably the biggest technology firms – Microsoft, Google, Amazon and Facebook – between them purchase more than $1 billion worth of networking equipment each year and are all heading away from traditional methods of building networks to adopt the SDN model.

As I will explain at my IET lecture, the transformation is being driven by the confluence of two key trends. The first is the growing need for companies that own and operate large networks to customise and differentiate their network to best serve their needs. The largest technology platforms, as well as carriers such as BT in the UK, and AT&T in the US, are looking to take control of the software that defines how their network behaves.

The second trend is the growing popularity of merchant switching silicon. Analogous to the arrival of the low-cost microprocessor that revolutionised the computing industry and put a PC on every desktop, merchant switching chips combined with third-party software threatens the ability of incumbent networking equipment vendors to differentiate. But for an operator, owner or user of a network it promises more competition, more innovation, and faster iterative progress.

Common patterns

Recent history shows common patterns in the way technology matures. The first programming I did was on VAX and IBM computers, the mainframes employed by university administrations the world over. At that time, IBM was to the computer industry what Cisco is to the networking industry today.

IBM was building mainframe computers based on microprocessors it manufactured in-house. These microprocessors were controlled and managed by an IBM developed operating system (OS), which ran IBM’s proprietary applications. This was the vertically-integrated and closed model of computing that endured up until the mid-1980s when Motorola and Intel introduced the first low-cost, off-the-shelf microprocessors.

By today’s standards, these microprocessors were not very sophisticated. Nevertheless, they came from somebody else other than IBM and enabled development of much smaller and cheaper computers. Significantly, in order to use them, you had to be able to programme them. Since Motorola and Intel did not want to get embroiled in writing software, they approached Paul Allen and Bill Gates and asked them to build a small OS that could run on top. Thus Microsoft was conceived.

Predicting the future using the past

The emergence of the low-cost microprocessor, and Microsoft’s rudimentary OS, led to more operating systems and applications being developed. This ‘open interface’ between what is known as ‘commoditised hardware’ and the software turned the mainframe industry on its head and the trend exploded into a computer on every desktop.

Conversely, networking technology has proved a laggard. OEMs of networking equipment have continued to sell vertically integrated and closed boxes. So why has this model survived for 25-30 years beyond the paradigm shift that occurred in computing?

Part of the answer lies in the way networking technology and the internet has evolved. It was important for these boxes (switches and routers) to be simple and self-contained in order to offer a ‘plug and play’ experience, so that people all over the world could build pieces of the internet. This proved a wonderful engine of growth.

But as the internet grew, the people running the networks became much more sophisticated. The boxes themselves became substantially larger and significantly more complex, such that a large router today has the same shape, size, and power consumption as a mainframe. Moreover, the technology inside all comes from one company and, like mainframes, is vertically integrated. Given the hundreds of billions of dollars at stake in the networking industry, there has been a strong vested interest to keep it this way.

Exposing the myth

The advent of SDN is exposing the myth perpetuated by networking OEMs that it would be disastrous to split the components within routers and switches apart. Crucially, it is turning an industry that was vertically-integrated and closed into one that is horizontal and open.

SDN is when the software is separated from the hardware that forwards packets. The software runs on regular servers, which can be programmed through an SDN controller using an industry standard control protocol such as ‘OpenFlow’. SDN software can control any OpenFlow-enabled network device from any vendor – including switches, routers, and virtual switches.

>See also: The software-defined data centre is happening today: Eschenbach, VMware

Rather than having to manage groups of devices from individual vendors, IT can use SDN-based tools to quickly deploy, configure, and update devices across the entire network. SDN also automates many management tasks that are done manually today, reducing operational overheads, decreasing network instability introduced by operator error, and supporting IT-as-a-Service and self-service provisioning models.

Moreover, SDN adoption accelerates business innovation by allowing IT network operators to literally program – and reprogram – the network in real time to meet specific business needs and user requirements. By virtualising the network infrastructure and abstracting it from individual network services for example, SDN and OpenFlow enable users to tailor the behaviour of the network and introduce new services and network capabilities in a matter of hours.

Back in control

The computing revolution analogy could well be a strong indication of what will happen next. Going back to the old IBM model in the mid-1980s, this was not sustainable because it was moving too slowly. The forces of competition and the need for more computing power meant there was no way IBM could have sustained the necessary level of innovation by itself.

In 1985 – the year Cisco was established – it was only natural Cisco would make boxes that had hardware and software, and features such as the routing protocols that decide where the packets go. But now, with the availability of merchant silicon, the manufacturer of the box largely becomes irrelevant, just like the low-cost microprocessor did in the computing industry. Likewise the rows of servers in a data centre are now white boxes based on Intel chips and a Microsoft or Linux OS, rather than the big brands of server technology preferred previously.

The same phenomenon is now occurring in the networking equipment industry, where vendors had up until recently been holding network operators and carriers to ransom. SDN and merchant silicon puts network operators and carriers back in control.

Thrilling ride

The impact of SDN is evident globally. Japanese carrier NTT announced three years ago that its entire global infrastructure will be based on SDN technology, with companies such as Amazon, Facebook, Google and Microsoft following suit.

>See also: 10 tips for IT pros in a software-defined world

These companies are run by software engineers – the businesses they run and the services they provide are software driven. SDN and merchant silicon enable them to build and control networks that are cheaper, better fit for purpose, faster and more reliable.

Google for example, has built its entire wide area network (WAN) that interconnects its data centres on SDN. The project has been running for two years and has enabled Google to test and deploy new applications much faster and realise significant benefits in terms of guaranteeing capacity for traffic flows and reacting faster to link failures.

OpenFlow not only makes it easier for Google to manage large volumes of traffic on the WAN, it also makes it cheaper. Since 2005, Google has not bought a single piece of vertically integrated networking equipment, preferring instead to purchase white boxes and write the networking software itself.

Earlier this year, AT&T, the biggest purchaser of network equipment in the world outside of government, provided more details on ‘Domain 2.0’. This is the vendor portion of its new strategy for building the network of the future – a ‘multi-service, multi-tenant platform where equipment is flexible and not specialised for a single function, and where the design taps into technologies such as network function virtualisation and SDN.

According to John Donovan, senior executive VP AT&T Technology and Network Operations, this strategy will see AT&T change everything about how its network is built, how it buys its network equipment, and the software that powers it – and its operations and culture too. When asked for the singular central reason why AT&T is moving towards SDN, Donovan responded: “Because there’s no army that can hold back an economic principle whose time has come!”

This is the context in which the SDN revolution is taking place and it needs to be shared with a wider audience. This is why I have chosen it as the topic of my IET lecture. With the revolution gathering pace and some trying to cling to the past, given the amount of money and control at stake, it’s time to hold on to your hats as this is going to be a thrilling ride.

Nick McKeown will be delivering the IET Appleton Lecture on ‘Software Defined Networks and the Maturing of the Internet’ on Wednesday 30 April 2014, at the Royal Institution, London