For teams wanting to streamline their development pipeline, achieving continuous delivery is impossible without automation. When assessing how much automation you need, aim first to enable the product team to support small, rapid changes.
Small, rapid changes are easier to test, to fix when things go wrong, and to complete. They also enable more rapid feedback, which in turn leads to quicker business and product hypothesis testing.
Without sufficient automation, especially in repetitive tasks like builds, unit tests, and integration tests, the team spends too much effort conducting checks manually. This incentivizes bigger changes to avoid having to run through the process multiple times.
Bigger changes lead to an increase in the surface area of change. This increase makes merge conflicts more likely and makes it more difficult to figure out the root cause of any defects.
What is “sufficient” automation?
You can determine the appropriate level of automation for your team by assessing whether the team can handle the desired throughput of changes without sacrificing the stability your product requires. Research from DORA suggests four DevOps metrics for measuring throughput and stability.
When assessing sufficient throughput, DORA’s research points to optimizing the frequency that a team deploys to production and the lead time for code to make it into production. If a team is not able to meet minimum expectations for these two metrics, further automation is in order to help enable small, rapid changes. Examples of automation that can improve throughput are triggering builds in a CI pipeline upon pull request, improving test suite runtime, and automating deployments.
When assessing sufficient stability, DORA’s research shows that high-performing teams minimize the proportion of deployments causing failure in production and the time it takes to restore service after a failure in production. Examples of automation that can improve stability are triggering static code analysis on every pull request, setting up incident alerting, running health checks, and collecting telemetry.
It is uncommon for most organizations to have data on the four DevOps metrics readily available. So, this kind of analysis can be done qualitatively at first. Simply ask: do we have concerns with meeting throughput or stability expectations? If the answer is yes, then further automation is likely in order.
Every industry, organization, and team has unique constraints, but the State of DevOps report includes data from high performing teams on the DevOps metrics.
Automation helps teams trade off throughput and stability
For teams meeting minimum throughput and stability expectations, where do you go next? Once you have achieved the minimum baseline, the team can strategically trade off stability and throughput.
For example, if a team is comfortably meeting its service level agreement of 99.9% uptime, should it continue optimizing for further stability? Likely not. The team can afford to take more delivery risks and focus on throughput.
This is one of the promises of “sufficient” automation: the safety to take on risks. The key here is not to sacrifice throughput or stability to the point where they fall below the defined minimum thresholds. This can place the business at risk of falling behind competitors or having an unreliable product.
CI/CD automation places priority on predictability
For all of this focus on automation, there are important human factors that also impact a team’s throughput and stability. CI/CD automation should support the team’s ways of working, making steps in the team’s process more predictable.
Often the most visible information radiator of the team’s process is its project board. Automation can play a key role in enforcing the entry/exit requirements for each phase in the team’s board. The automation you implement should support code moving through from one phase to another by ensuring that it meets the expectations of the teammate picking up the work, which minimizes rework and defects.
Note: Defining and assessing worthwhile automation is a separate exercise from conducting a process review. My colleague Rob O’Brien discusses process reviews in his post, How process bottlenecks disrupt continuous integration and development.
CI/CD depends on automation
As with any process, automation can help reduce the chance of human-introduced error. Especially in the case of CI/CD, it additionally speeds processes to support shorter feedback loops and greater efficiency. Introducing automation to your CI/CD pipeline can lead to greater team productivity and happiness. It also benefits the business by maintaining product reliability and creating opportunities for growth.
Michael Wytock
