This post is from the Collabnet VersionOne blog and has not been updated since the original publish date.
SAFe 4.5 DevOps metrics - scale continuous delivery with the Scaled Agile Framework
The latest release of the Scaled Agile Framework®, SAFe® 4.5 includes focus on DevOps and the continuous delivery pipeline. What is helping large enterprise organizations succeed at continuous delivery, DevOps, and SAFe? Read on.
SAFe’s scalable DevOps approach
DevOps supports the entire flow of value. The CALMR approach speaks to Culture, Automation, Lean/flow, Measurement, and Recovery. Culture is a shared responsibility across the organization from development to operations to security - everybody who's building anything needs to be part of this. Automation is very important, making sure you automate the entire delivery pipeline. You can’t do just agile or just DevOps. You have to merge the concepts together so lean/flow is an important part of the CALMR approach. “If you are building things in large batches and have very advanced continuous deployment automation, but are still developing large batches, it doesn't help you.” said Inbar Oren, SAFe fellow, principal consultant at Scaled Agile, Inc. If you want to release more often, you need to be able to recover and monitor risk.
Hypothesis-driven continuous delivery pipeline
A central change in SAFe 4.5 is the concept of a hypothesis-driven continuous delivery pipeline. When it comes to DevOps, a lot of people just focus on delivery pipelines - but Scaled Agile feels that continuous delivery starts much earlier in the software development lifecycle. Inbar Oren shared that you need a holistic view from exploring new ideas and how customers want to integrate those ideas into larger and larger things. This is bigger than just automation or having a continuous build process. We need to look at it as a full loop of hypothesized build, measure, and learn.
Continuously explore customer needs
Measuring continuous exploration is the first of four elements of the continuous delivery pipeline. Product managers need to talk to internal people, system architects, business owners, and customers as they do market research. Scaled Agile puts a lot of emphasis on the lean startup cycle and Lean UX cycle at this phase. This concept of fostering innovation with the lean startup cycle is crucial to the understanding of SAFe 4.5.
Foster innovation with the lean startup cycle
You need to enable decision makers at the portfolio program level to take their efforts, break them into minimal viable products, build them, integrate them, deploy them, release them, and measure it to see if it is actually providing results from adding features.
Increase experimentation over time
SAFe 4.5 now refers to the epic value statement as a hypothesis statement. The hypothesis statement describes the purpose of an epic and how it is going to be measured. The larger and more complex your organization is, the harder it is to have an idea, develop a hypothesis, get it to market, and test the hypothesis. Inbar Oren explained that they see a lot of companies that have a hypothesis but don’t acknowledge it or only measure it once a year and that they want to change this and see more hypotheses tested over time.
Continuous integration of components, systems, & solutions
Inbar Oren shared that from Scaled Agile’s prospective, continuous integration is bigger than just integrating on a dev machine or at the team level - it's integrating at all the different levels of the organization. Far too often, there are changes or commits made to the body of code that can't be tied back to specific business value (backlog item, epic, or feature). As we move faster, we need to get better at audit and compliance and being able to be sure about what changes have been made in software. If we don't connect every single commit back to business value, it becomes really difficult to measure the flow of business value once we've converted commits into binary objects or into some compiled object.
Continuous integration metrics
Traditional continuous integration metrics apply:
- Build length
- Build fails
- Testing reliability
- Number of commits per developer per timebox
- Rogue commits
More continuous integration metrics apply at scale:
- Feature cycle time from backlog to approval on staging
- Cycle time from commit to system demo
- Cycle time from system demo to solution demo
- Cadence of demos
The focus of SAFe DevOps metrics is to measure how long it takes to develop an idea and deliver it to your customers and the cycle time at different stages of the software development lifecycle.
Release on demand
Scaled Agile suggests focusing on releasing on demand as opposed to releasing anytime. There may be business reasons that you can’t release but you can always deploy. Start looking at your solution to see what you can decouple into smaller chunks of value that can be delivered at different times, or subsystems that can be released, or security fixes that can be done. You will find you can actually do more deployments that you may think. Deploying more often ensures that your software is tested and ready to go.
Measure the entire value stream
How do you measure an entire value stream? Oren suggested using a program kanban board to visualize the flow of work from idea to completion.
Identify bottlenecks with cumulative flow diagrams
We've leveraged cumulative flow diagrams in the agile world for quite a while, but we are now bringing the cumulative flow diagram all the way to the definition of done, or at least to delivered value, to identify bottlenecks and opportunities.
Map pipeline efficiency
In DevOps, efficiency is a strong indicator of bottlenecks and where there are opportunities. It is a combination of touch time and wait time. If you know cycle time at any given phase, then to calculate efficiency you only need to understand either wait time or touch time and you can derive the others.
Scaled Agile Framework and SAFe are registered trademarks of Scaled Agile, Inc.