3 Performance Measures That Can Help Reduce DevOps Wait Time

Pretend you are a newly/planned user story or fix just starting your journey to production (wait… what?). You are very excited because you know you are loaded with potential value that your business really needs. You really can’t wait to be deployed! However, you must make your way through the DevOps process before your end/users will get to check you out. This means you have to get coded, tested, approved, and deployed. Unfortunately, I have some very bad news for enterprise backlog items… A recent report from lean101.org suggests you’ll spend 95% of your journey to production twiddling your thumbs — simply waiting. Even within above/average enterprise organizations, less than 5% of the total commit/to/end/user travel time is spend adding value. Worse, this number actually dips well below 5% as the average batch size of production deployments increase. In other words, bring a book, a pillow, and a toothbrush. The vast majority of your DevOps journey will be spent simply waiting for someone or something to move you along your way. This statistic is why virtually every organization focused on developing software is frantically trying to increase velocity by reducing delivery wait times. In fact, reducing wait time is a fundamental goal of DevOps. What if you could actually track each backlog item end/to/end through each and every stop of a value stream map? Furthermore, what if you knew exactly how long each backlog item spent at each phase or step of the DevOps value stream map? These DevOps flow measures and the ability to produce them is quickly becoming a key success factor in the quest to increase delivery velocity and reduce DevOps wait time. This post will describe three ways to measure DevOps flow along with the fundamental capabilities you need to have in place to make these measurements possible. Don’t Our ALM Tools Already Measure Flow? VersionOne’s Lifecycle solution can measure flow by manually tracking the status of a backlog item through to the definition of “done”. This is awesome and very important; however, we lose direct visibility to business value once a backlog item is converted into code and binary objects. From that point on, we are left trying to manually translate binary code into the incremental new business value it carries. This translation process is difficult, time consuming, and very imprecise. As a result, most organizations have little high/fidelity data describing how business value flows between development and end users once backlog items are converted into some form of machine language. DevOps flow measurements track the business value that is carried in that machine language all the way through delivery to end users. DevOps Flow Measures & Metrics DevOps Flow metrics are a rapidly emerging collection of value/based DevOps performance measurements that can help organizations visualize and measure the flow of business value in the form of incremental software. Flow measures typically begin with the first commit in a source code repository and track value all the way to end users. I call them “value based” because instead of measuring the DevOps machine itself — as is the case with most of our current DevOps metrics — they actually measure the machine in terms of how value is flowing through it. Leveraging value/based measures to evaluate DevOps performance is a crucial starting point for achieving incremental and sustainable progress toward continuous delivery / especially within the enterprise. Point/in/Time DevOps Flow Metrics Track Delivery at a Glance by Package or by Portfolio Item with VersionOne Continuum for DevOps Point/in/time flow measures show the current distribution of business value, in the form of backlog items — user stories, defect fixes, features, and epics — across all phases of DevOps. These flow measures clearly outline which backlog items are in any given phase of delivery at this exact moment. Here are a few important benefits of point/in/time flow metrics:

• Development teams: see exactly what business value is “baked into” any group of deployable artifacts

• Risk and compliance: document what is being delivered and demonstrate that all controls have been implemented and successfully passed

• Release managers: demonstrate that any given potential deployment has met all the quality standards required to achieve release and help ensure that the proper binary artifacts are always selected for deployment

• Product managers: provide insight into how much of any features or epic has been delivered to end users and how much value is currently parked in each phase of the value stream map

Time/in/Phase DevOps Flow Metrics Time/in/phase flow metrics show the lead time at each step of a value stream map — all the way to the end user. These metrics illuminate bottlenecks in the delivery process and provide critical feedback as workflows evolve, manual processes are enhanced, and new automation is introduced. While important to all stakeholders in a software value stream, time/in/phase measures are particularly beneficial to CIOs, development leaders, and any role responsible for driving velocity. Three benefits of time/in/phase metrics:

• Establish current performance baseline: These measures provide a baseline of DevOps performance, empowering you to iteratively optimize your DevOps pipelines. Absent this benchmark data, there will be no clear definition of the problem and, more importantly, there will be lots of opposing points of view regarding the next best opportunity for DevOps optimization.

• Highlight performance bottlenecks: Time/in/phase flow metrics highlight performance bottlenecks, enabling teams to prioritize efforts and initiate deeper root cause analysis to better understand why value is not flowing as freely as it could.

• Measure actual time/to/value: Since no value is delivered until it is in the hands of actual end users, it’s important to measure cycle time through value delivery — regardless of any definition of “done”. Time/in/phase metrics capture elapsed time through the entire development and delivery process, providing a precise measure of time/to/value.

• Measure all aspects of DevOps: Perhaps the best thing about implementing a flow/based measurement program is that it gets to the very core of DevOps, minimizing the friction in a DevOps value stream. It doesn’t matter if the friction comes from people, process, or tools — flow/based measurement will find it.

Deployment Metrics With each deployment event, we are delivering new and incremental business value. Tracking key attributes of each deployment can help us better understand how value is flowing to end users. Here are some examples of key deployment metrics:

• Batch size /  In DevOps, batch size describes the amount of potential business value carried with each deployment event. Since batch size is a key factor driving wait time, reducing batch over time is often a top priority among DevOps teams. I favor using the number of backlog items (user stories and defects) as the measure of size. I’ll discuss the pros and cons of using backlog items as the “atomic unit of business value” in an upcoming post.
• Deployment Time  /  Deployment time is typically measured as the number of days between each deployment. This measure is also a key factor in reducing delivery wait time. DevOps teams seek to reduce deployment time as the DevOps process becomes more efficient over time.
• Batch Distribution / Batch distribution is another way to describe the value payload contained within each deployment.  This measure describes the ratio or percentage of business value across multiple value types.  While value types can vary by organization and team, here are some common examples:
  • New Capability
  • Defect or Fix
  • Technical Dept
  • Legacy Migration
  • Configuration

Conclusion Implementing a DevOps performance measurement program based on DevOps flow metrics and data will enable the automated and highly precise measurement of end/to/end cycle time yielding better decisions and smarter investment in people, process, and tools. Having clear visibility into end/to/end cycle time will dramatically improve your ability to incrementally optimize the value you deliver to your customers. I hope this has encouraged you to take a deeper look at how your organization is tracking the value that is being delivered to customers. These three types of value/based DevOps performance measures are just the tip of the iceberg, so if you want to learn more, check out the other blog posts in this series:

• How Measuring DevOps Performance Increases Enterprise Agility
• Measuring DevOps Performance Using a Value/Based Approach
  • 4 Basic Building Blocks Needed to Measure DevOps Flow

Continuum is a trademark of VersionOne Inc.