A strong DevOps strategy can help to break down the silos between business and IT, necessary to maintain productivity, efficiency and security in this new age of remote working, which is why Information Age has produced this DevOps guide
Ultimate guide to DevOps
This ultimate guide to DevOps will cover 13 topics in the following order:
- What is DevOps?
- Why do we need DevOps?
- How DevOps works in enterprise
- DevOps leadership
- Establishing a strong DevOps pipeline
- 8 steps to DevOps
- What is DevSecOps?
- The DevOps challenge
- Agile vs DevOps
- What are the most vital roles in DevOps?
- How to build a DevOps team
- Top DevOps interview questions
- The most important DevOps tools for tech workers
“DevOps has been increasingly adopted to break down silos between business and IT as organisations move to a product-centric model,” says Peter Hyde, research director at Gartner.
“If we don’t have everybody in the whole product development and support teams working together, we create barriers, we slow down development, we slow down the support for our customers. The most important metrics we now have is customer lead time; how long from concept commitment to delivery and production for our customers. We need everyone working together on that. We need integrated teams to deliver value,” he continues.
What is DevOps?
Hyde’s favourite definition of DevOps is “the practice of operations and development engineers collaborating throughout the entire service lifecycle, from initial concept through the development process to production support. They we need to understand each other to better work together”.
The notion of DevOps is relatively simple, although putting it into practice is somewhat more complicated, requiring a level of collaboration, communication and leadership which many organisations struggle with.
DevOps requires a culture change as much as anything else — which is something organisations going through digital transformation efforts will recognise as a significant hurdle.
AWS defines the term pragmatically as “the combination of cultural philosophies, practices and tools that increases an organisation’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organisations using traditional software development and infrastructure management processes. This speed enables organisations to better serve their customers and compete more effectively in the market”.
Annie Andrews, chief technology officer at Microsoft partner Curo, defines DevOps as “a culture that has a number of things underpinning it. If you talk to 10 different people, you’ll get 10 different answers to the question, which means that when you’re explaining what is DevOps, you’ve got to be really careful that you’re giving a particular slant on it”.
In short, DevOps accelerates fluidity between developers and operations, which leads to faster delivery times when it comes products and applications without sacrificing quality.
Why do we need DevOps?
Businesses by now know that they need DevOps, but they’re not entirely sure how to implement it.
They need DevOps “simply” to compete with an increasingly congested marketplace, to streamline internal business processes and to get the highest quality products out to their customer base as fast as possible.
“To drive competitive differentiation,” reiterates John Rakowski, now head of product marketing at pre-paid corporate credit card provider Soldo, but that’s from the consumer experience (UX) point of view, rather than internally.
“As consumers are now ‘always on’, businesses must perfect the digital user experience through the continuous delivery of reliable services, whether that be traditional web, mobile applications, IoT devices or wearables. Development and operations combined enables organisations to develop better, more consumer-focused applications with a higher degree of consistency,” he says.
How it works in the enterprise
Before implementing DevOps in enterprise, leaders must first identify why they want to adopt it. Here, the old adage applies: don’t apply technology for technology’s sake.
“You have to know why you’re trying to do it. What is it you want? Do you want repeatable quality? Do you want cheaper or faster deliveries? Do you recognise a need to modify the architecture,” asks Gordon Cullum, technology director at Axiologik Solutions.
How DevOps works in the enterprise depends on the journey — frustratingly, perhaps, there is no simple answer.
The key takeaways for business leaders should be:
- Don’t underestimate the cultural change required (people have to buy into the idea, similar to digital transformation)
- Don’t rely too much on heavy documentation (you’re not going to know everything up front)
- Approach risk proactively (don’t be afraid of change)
If a business then decides to implement DevOps within teams, from a process and method point of view, then these questions must be addressed:
- Is your architecture able to support it?
- Is a leadership roadmap in place that creates the environment necessary to start delivering fast, high quality, automated deliveries?
Leadership
“Everything is holistic and needs top-down support,” says Hyde. “Businesses need someone from the top level explaining: what is our business driver? What is our vision?
“They also need middle management and practitioner support and buy-in; they need leadership to communicate what DevOps means to them, does it change the political hierarchy?”
Establishing a strong pipeline
The perfect DevOps pipeline consists of a repetitive cycle of the following stages: plan, code, build, test, release, deploy, operate and monitor – all as a continual process.
However, too many organisations have to choose between speed of development and poor-quality code. To create a strong DevOps pipeline, IT leaders should use automation.
- Prepare a roadmap
- Gather user stories
- Split the work into small pieces
- Use automation where possible
- Use AIOps (artificial intelligence operations) where you can to reduce routine, highly repeatable tasks slowing down the delivery pipeline
- Stay compliant
- Employ standardised tools and technologies to prevent non-compliant apps from reaching end users
- Incorporate backup – ensure any code changes are secure
- Integrate data – weave database development into any DevOp plans
8 steps to DevOps
- Business goal. What is our business goal moving to DevOps? Sometimes it’s speed of delivery. How do we do early delivery, frequent delivery? Once businesses have that vision, they can start to drive that down and understand how we can implement DevOps
- Collaboration. We might do that with ambassadors or operational liaisons
- Customer outcome. How do we focus on achieving what our customer needs? And how do we work with our customer better, so that they can understand how things are changing?
- Automation. How do we automate through the pipeline? This is the full pipeline now, not just development. It’s taking the whole, from initial planning to creation, verification and pre-production release; all the way through the lifecycle and back round again, so we can bring all that great data back into our planning
- Compatible tooling. If an organisation uses different tooling, it begins to create barriers again
- Real-time monitoring. We need to use much better monitoring tools to help react faster to issues
- Deliver continuously. How do we get our continuous integration/continuous delivery (CI/CD) pipeline to work in a modern way?
- Retrospectives and reviews. How do we do workshops together as a full development value stream, include everyone and really update process piece by piece in an iterative way to increase capability?
What is DevSecOps?
Developers and those in IT operations need to take a more holistic approach to security. The stakes could not be higher.
DevSecOps, as the name suggests, is the combination of development, security and operations.
“Security must be integrated into the agile development process. By doing so, organisations will be able to address security threats more effectively and in real time,” wrote Rusty Carter, now chief product officer at SafeguardCyber.
“Making security a shared responsibility between development, IT and security teams should help change the perception that security is a burden and slows down the agile process — in addition to sensitising the entire team to the need for speed and agility to deliver new solutions to market.”
The DevOps challenge
The biggest DevOps challenge to a successful implementation concerns outdated IT architecture, which can’t support the new, agile way of working.
The need to modernise IT infrastructure is relevant to the larger organisations that are steeped in legacy — ironically, those which would benefit from DevOps the most. Fortunately, the business case for IT modernisation goes beyond DevOps and should be one of the priorities for any organisation wanting to compete in the era of disruption. IT needs to be embedded into the business community.
Luca Ravazzolo, product manager at InterSystems explains: “As the cloud world evolves, we will see the development of more intelligent tools that will allow us to follow up DevOps processes with more discipline and become more efficient. We have seen this in the last few years with the rise of cloud offerings which try to bring together the coding and building of a solution. This will help provide solutions for issues such as testing, deployment and security.”
Agile vs DevOps
They have their differences, but both disciplines are pulling in the same direction — they share a similar philosophy.
Traditionally, development and operations teams — separately — would work on relatively heavy waterfall-style projects, but as soon as these started to attract users, it became clear the problem wasn’t being fixed — hence, the rise of DevOps.
DevOps emerged out of frustrations between development and operations, whereas Agile was born out of teams not being able to deliver what the customer actually wanted.
It is the bridge that unites these Agile developers with their IT operations counterparts – the folks taking care of applications in production including service management, performance management, monitoring, security and incident management.
With Agile, internal project managers doing date-based delivery used to plan three-year projects for building software. “In three years, we know where this software is going to land,” they would say. But, the industry has learned that software development doesn’t work like that.
Initially, DevOps was more focused on solving internal organisational problems, whereas Agile had a much more outbound aspect to it. But now both are reflected internally and externally.
Rob Greenwood, founder and technical director of Steamhaus says: “Both are structured, but lightweight, with the focus on efficiencies. They also both follow iterative processes, which contrasts with waterfall — where the expectation is that a fully functional product is delivered at the end.
“With Agile, tasks are prioritised and then organised into short sprints or ‘time-boxes’. This sprint approach works well with DevOps’ cyclical processes – and a toolset designed to offer various reusable components. With waterfall, however, you would restrict the key benefit of DevOps – namely, continuous delivery.”
Although DevOps and Agile were developed as standalone concepts increasingly they have become fused together.
David Huntley, continuous improvement director at project manager Distributed, says of the two: “DevOps and agile are not in competition, they work hand-in-hand to ensure everyone is working together as smoothly as possible. There are differences between the two concepts though.”
Huntley explains: “Agile encompasses everything upstream of development – it’s about working more closely with the wider business in order to develop a product that is fit for market. Meanwhile, DevOps is about working downstream with development and operations teams to ensure that new software works in production.
“In other words, Agile is about getting software out the door quickly and DevOps ensures it’s done safely and reliably, which is why the two complement each other so well.”
However, the fusion of DevOps meets Agile is alien to most large enterprises, which are used to strict information technology infrastructure library (ITIL) methodologies for their run stage.
Chris Astley, head of connected engineering at KPMG UK, says: “They will find their operations activities appear disorganised and lack structure. But this isn’t necessarily true – it’s more that the normal structures IT leaders look for have been replaced with ones they don’t recognise.”
What are the most vital roles in DevOps?
The number of job ads for IT professionals more than doubled last year compared with 2020, with more than 12,800 vacancies posted, according to recruitment consultancy Robert Walters.
And nowhere has this been more keenly felt than in DevOps teams.
Zoë Morris, president of recruitment firm Jefferson Frank, says: “We’re already in the middle of a massive skills gap, so finding DevOps specialists is difficult for almost any organisation.”
What are the key DevOps roles?
DevOps evangelist is responsible for ensuring the success and implementation of all DevOps processes and team identity.
Code release manager is the project manager role in a DevOps model, who must also understand how and when to leverage Agile ways of working.
Automation architect creates processes that use automation to help reduce manual tasks. They are responsible for creating a more efficient process and finding the right tools to use and integrate within a DevOps model.
Experience Assurance Expert (XA) is responsible for creating a smooth user experience of the final product, making sure the end product not only works correctly and has the right features, but also that it’s easy to use.
Software developer/tester is the one responsible for writing the code, and in a DevOps setting, the software developer/tester also performs unit testing and deployment, as well as ongoing monitoring. This is a bit more of an expanded role compared to the traditional developer, who is mostly concerned with just writing code.
Security & compliance engineer (SCE) is responsible for the overall security of the system. In DevOps, the SCE works right alongside development integrating their security recommendations while the product is being built rather than during post-production.
All of these roles need to keep abreast of what’s going on in the fast-moving world of DevOps.
“You need to be constantly reading tech articles, constantly looking at releases and updates for the likes of AWS. You really need to be kept up to date with what’s going on, because there are new tools coming out every day. If you don’t know about them, you’re not going to use them and you’re not going to deliver the best DevOps process for your company,” according to Mark Quinn, co-founder and CTO of Circl. Gaming.
How to build a DevOps team
Building an effective DevOps team — filled with DevOps engineers (among other roles) should be a priority for organisations.
“When you think about building out a team that manages DevOps, the focus should not be on the tools but instead on the understanding (a) the discipline, and (b) the processes,” explains Utsav Sanghani, director global product management at open-source software company Suse.
- Focus on discipline and process
- Understand the release velocity and release agility
- Champion collaboration
- Change the culture
- Don’t forget the data
“It all starts with collaboration,” says Hyde. “How do we get development and operations collaborating together? How do we start to understand each other’s languages, each other’s challenges and the pain points?”
Top DevOps interview questions
Here are the top DevOps interview questions:
• What DevOps tools have you worked with? Why is DevOps important and what does it represent?
• Describe what DevOps has helped you achieve — both in a work environment and personally.
• Are you a team player? What is your motivation for working in DevOps?
• What skills do you possess? What use cases of previous projects can you provide?
• Tell me about past failures or challenges and how you’ve overcome them from a software development and/or technical operation side? Are you an effective communicator?
Find the answers to these questions here.
“I’m looking for people who when they come across a problem, they take on the responsibility of solving it,” says Nigel Kersten, VP of ecosystem engineering at Puppet
The most important tools for DevOps tools for tech workers
There are so many DevOps tools out there to help you deliver, it can feel overwhelming. Github or Gitlab? Openshift or Kubernetes?
DevOps works in the cross-functional mode, involving various tools of various kinds and purposes, instead of just a single tool. These tools are also known as DevOps toolchains.
The tools help throughout the software production lifecycle, including development, management, and delivery.
The organisation that uses DevOps practice coordinates these tools and fits each one of them into one or more production activities:
- Automation
- DevOps pipeline (CD/CI) tools
- Version control
- Configuration management
- Testing
- Monitoring
Automation
Jenkins
Jenkins is an open source and free automation server that helps automate software development processes such as building, facilitating CI/CD, deploying, and testing. This tool makes it easy for teams to monitor repeated tasks, integrate changes easily, and identify issues quickly.
Docker
Docker is used by more than 11m developers worldwide. As a DevOps tool, Docker helps developers to build, package, and then deploy code with ease, eliminating mundane configuration activities and fosters effective team collaboration.
Maven
Maven is one of the important DevOps tools for building projects. Apache Maven is designed to manage reporting, documentation, distribution, releases, and dependencies processes. It eases the build and monitoring process through automation and maintains a uniform build process allowing for consistency and efficiency. Did you know the company name maven comes from the Yiddish, meaning an accumulator of knowledge?
Kubernetes
Kubernetes is an open-source production-grade container orchestration system to automate the deployment, management, and scaling of contained-based software. As one of the best DevOps automation tools, Kubernetes is useful for applications packaged with many containers. It allows developers to isolate and package container clusters logically for easy deployment to several machines.
DevOps pipeline (CI/CD) tools
Circle CI
No matter your DevOps team size, CircleCI speeds up your software development process. CircleCI enables you to develop and deploy high-performing software on your infrastructure or in the cloud at scale quickly. Used by leading brands including Spotify, Ford and Samsung, it now runs around 35m builds per month.
Version control
Git
One of the most widely recognized DevOps tools, Git is open source and free version control software. This distributed source code management (SCM) tool is used to track changes in files for software development and coordinate work effectively. Git tracks changes in any set of files being created by programmers developing source code together. It has been designed with speed, data integrity, and support for distributed, non-linear workflows (thousands of parallel branches running on different systems) in mind.
GitHub
GitHub is the world’s most advanced and largest development platform. It provides web hosting for software development as well as version control utilizing Git. Currently it is used by more than 56 million users and over 3m organisations worldwide. It provides the distributed version control of Git (see above) but also offers access control, bug tracking, software feature requests, task management, continuous integration, and wikis for every project.
Configuration management
Chef
Configuration management (CM) is critical to any system as it is the process by which changes in the system are tracked, properly implemented and controlled. Further, if not automated, CM can be laborious, resource draining, and prone to costly errors. It implements configuration tools for such repetitive administrative tasks such as version management, regulatory compliance, feature releases, and processes automation, etc.
Chef is an open-source cloud configuration management and deployment application that helps orchestrate servers in a cloud or just in a departmental data centre. Chef enables programmers and system administrators to work together. Instead of developers writing applications and then waiting on Ops staff to figure out how to deploy the software, Chef serves both communities. Chef offers a continuous delivery model by enabling effective and automated workflow.
Ansible
Ansible is a leading open source tool that provides simple but powerful automation for cross-platform computer support. Unlike more simplistic management tools, Ansible users (like system administrators, developers and architects) can use Ansible automation to install software, automate daily tasks, provision infrastructure, improve security and compliance, patch systems, and share automation across the entire organisation. Ansible allows you to configure not just one computer, but potentially a whole network of computers at once, and using it requires no programming skills.
DevOps testing
Selenium
If you are looking for a DevOps tool to automate your software testing, Selenium is a good bet. You can also use it to automate boring, web administration tasks. Selenium is an open source software released in 2018. It provides a portable framework to carry out the tests in addition to authoring functional tests with no need of learning a scripting language.
The rationale behind automation is to eliminate human involvement in certain tasks by using technology to automatically execute those tasks. Automation testing or test automation employs software applications, apart from the software being developed, to execute test cases automatically, compare outcomes, and report defects. This process is performed over and over to deliver high-quality software to end-users. Automation testing is an integral part of software development and is less prone to error.
Monitoring tools
Splunk
Software security is one of the prime concerns of any DevOps team. Using the Splunk tool, you can collaborate together on an incident and be aware of the rising security threats at the same time. The tool further gives DevOps professionals an option to mitigate such risks instantaneously using techniques such as file detonation, device quarantine, etc.
Nagios
Similar to Splunk, Nagios monitors applications and servers as well as your overall business infrastructure. Nagios alerts the users in case a particular fault occurs on the backend or any device fails. It also regularly maintains a performance chart and monitors trends to alert the user of a possible failure which may occur.
[emailsignup]
