If it only takes one person to change a setting to make a huge section of the Internet inaccessible, it’s time to address technical debt. The recent Fastly and Akamai outages, which together caused online gaming platforms, government websites, and news providers to go offline for more than an hour, have brought the challenges involved in managing the web’s infrastructure and software into mainstream awareness.
Technical debt as a concept has been around for a long time, and the two outages are very visible examples of a much bigger problem. The term describes the difference between the “price”, in terms of time, human resources, or technology investment, a technical project should cost in order to be completely futureproofed, and the “price” an organisation is prepared to pay when executing the project at the time.
Given we don’t live in a perfect world, every project will rack up small amounts of technical debt, for all kinds of reasons. They will eventually reach a certain level where they must be addressed, before a product develops flaws, is unable to be upgraded, or completely ceases to function as intended. As time goes on, small amounts of technical debt gradually mount up. This can eventually result in a large-scale incident, whether malicious or accidental in origin.
In my experience, I’ve found there are typically three main areas where technical debt builds up that IT and security leaders should be monitoring and factoring into their ongoing risk assessment.
Establishing a strong network monitoring strategy
1. Redirected investment
All companies change and shift their priorities as time goes on. They may choose to redirect budgets and resources to new projects and technologies, while older products remain part of the overall offering, but may be left to languish with less support or attention given than previously. But if clear end-of-life plans are not put in place, there’s a real risk of significant security holes appearing, with products created in outdated code which cannot be upgraded to work with latest version of operating systems or other compatible software.
The codebase of a product is put on hold as soon as investment in it stops, and it can cause issues with ongoing maintenance and management, as well as increasing the exposure to vulnerabilities and misconfiguration.
Investment decisions don’t only impact older products that are gradually moving into obsolescence. We also see this happening with products that haven’t even been fully released yet – an ideal development scenario might take significant time and investment, but a viable product can be created in a shorter timescale. Business priorities might take over, and pressure comes down to complete the product, even if the end result is not perfect.
Finding a suitable balance on the scale between perfection, appropriate functionality, and minimum viability is difficult, and customers or even development teams internally can find themselves being promised improvements once a project is complete, but then business priorities change and these never materialise.
Technical debt in this context can be managed by setting clear objectives and working closely with development teams to constantly manage expectations.
2. Physical technology
The second area where technical debt can accrue is with physical technology. It’s not uncommon to see data centres invested in, opened and then left without further management for up to four years. Operating with a mix of legacy and new services within an IT supply chain can leave a company in a precarious position.
Industries like financial services and healthcare rely on an array of data sources, microservices and other systems working alongside each other. It’s challenges with these integrations that can cause critical services to be knocked offline. Critical infrastructure is often built on OT (operational technology) that is proprietary in nature, which can be tricky to align properly with modern digital services. When you also consider the lengthy nature of software supply chains enterprises, governments and critical infrastructure relies on, and it’s not hard to see how large quantities of legacy and unsupported technology could cause serious damage in future.
Kaseya: the turning point for supply chain attacks?
3. People
Another huge source of technical debt, and perhaps the most important, is people. Software is an iterative product, and much of it has been developed over decades, by teams of workers with significant experience and institutional knowledge. These teams are also responsible for maintaining and managing older technologies and platforms.
But as business priorities change over time, systems built on older code can be neglected. Software development teams’ attention turns elsewhere, either by choice or force – which can create disenfranchisement among staff if not managed correctly. When access to and knowledge of older code resides only among a few people, we see potential insider threat risk of particular concern if software is being used to run critical IT infrastructure.
To that end, IT leaders must factor in succession planning into any strategic discussions they’re having. All workers eventually leave or retire, and if knowledge isn’t shared, you risk older systems becoming impossible to manage by newer employees. The importance of getting the basics right, such as applying updates and patches or managing configurations, never goes away, even for older systems.
Companies also need clear end-of-life processes, even for new products. It’s the last thing any team might want to think about, but contingency plans need to be put in place so that when organisational change happens, responses have been mapped out to address the potential impact on existing software and hardware. When it comes to future investment, legacy technologies can’t be completely left to languish, nor should new software only be launched with scalability in mind – there should be a plan for change and a clear idea of future upgrade paths and lifecycles.
Nothing lasts forever – software is no different. Technical debt builds up gradually over time but if not addressed, can create huge blind spots that leave organisations vulnerable to outages or worse. For the sake of future stability, leadership must take it seriously.
