Four steps to identifying the metrics that matter most to your value stream
To discover which metrics have the strongest effect or relationship with value creation, here are 4 steps to identify them.
Value stream management (VSM) demands a radical perspective transformation for business operations. While it continues the advances begun by lean manufacturing, agile, and DevOps, the key difference is that all activities are focused on generating more value through individual value streams. Ideally, priorities or goals that don't directly contribute to value creation will be eliminated or realigned.
The perspective change comes when all business areas begin to work in concert towards collaboratively improving outcomes, thereby, delivering more value more often to customers. Value-indicating metrics are a key component in achieving the desired transformation. But how are these revealed?
There are general guidelines for choosing appropriate business metrics, such as the following curated by DevOps & Big Data specialist Vladimir Fedak. He reminds business leaders to choose only metrics that are:
- Measurable — Metrics have numeric values. Subjective qualities are not metrics unless they have been quantified according to a strict rubric.
- Relevant — Metrics must measure something immediately or contextually relevant to business activities and goals.
- Incorruptible — Metrics must be free from bias or arbitrary self-selection among teams.
- Actionable — Metrics must offer feedback that prompts action, revealing opportunities for improved processes, workflows, policies, or tool selection.
- Traceable — The metrics must be attributable to specific sources through granular drill-down capabilities, allowing leaders to trace root causes of metric performance directly.
General guidelines are helpful, but one critical component of VSM is the discovery process of determining which metrics can reveal insights about value creation and how to stimulate it further.
To discover which metrics have the strongest effect or relationship with value creation, here are the steps former VP Platform Strategy at Digital.ai Andreas Prins recommends:
- Extend your view to post-deployment
- Discuss VSM initiative success with internal business partners
- Identify bottlenecks, and take them seriously
- Map IT activities, and compare them to company objectives
1. Extend your view to post-deployment
Monitoring development with a narrow view misses an important feedback loop: the things your customers would want to tell you in order to improve your product. Developers may focus on hypothetical performance or user experience without understanding actual performance or user experience (UX). To understand what the customer experience is like, look at things like security, performance, availability, and user behavior in production to determine how the product is actually used. The product can also be subjected to the same performance tests that ensure quality during the development stage.
Looking past deployment provides extremely valuable input to everyone, especially the product manager and the core feature development teams. Gathering metrics during production allows these teams to start making different decisions to achieve better user outcomes.
Actively monitoring the live product also provides vital context. The gleaned performance and UX metrics can be connected to qualitative user reports to form a complete picture of the value the product is delivering to end-users. For example, a user report or review may complain about the following: "The app takes too long to load when I do action XYZ". To complete the feedback loop, DevOps leaders can monitor metrics like server response times or app submission response times during testing and in production.
The overall goal is to focus on what the customers are saying based on their own experiences. Otherwise, development may just be focused on what's next instead of focusing on improving the existing product in a way that's measurable to customers.
2. Discuss the success of VSM initiatives with your internal business partners
The discovery of key performance indicators comes from collaboration. Start conversations between the product manager, business line managers, and others with direct accountability for product performance and CX. With these individuals in the same room (or in the same virtual call), they can piece together the feedback indicators they find most valuable for performing their job role.
Look for differences in perspective or misalignments that can impact focus towards a singular goal or outcome. For example, development teams know that their performance looks best when they can commit more code faster and deploy new releases more often. But security teams know that a single breach or leak can seriously impact their company’s reputation and revenues. They may want development to, ideally, slow down and test more to make it less likely that a major defect escapes into production.
Every discipline involved in the definition and creation of an initiative has its own view or perspective. Identify these differences, and look for ways to flatten them into alignment.
Consider differences in perspective for when value is created. Going back to the above example of push/pull between security and development, individual engineers consider value creation as the moment they commit code. For infosec, value is created every day there is not a breach or an incident.
These conversations help various business areas map their own perspective and determine where to go from the present position to reach the transformative goal. In the case of development and security, security measures can be implemented further left alongside greater coverage and more frequent test coverage. A combination of security by design, best practice education, and test automation can potentially achieve alignment between the two camps while adjusting the definition of value creation for each.
3. Identify bottlenecks, and take them seriously
Bottlenecks in value delivery tend to have a severe negative effect on value creation. These bottlenecks are often related to team handoffs, gated approvals, or areas with extensive backlogs, like InfoSec reviews
To find bottlenecks, dissect your own processes. Map them for metrics like cycle time, commitment frequency, sprint burndown performance. Identify outliers, which can mean individual steps taken or individual team performance.
Former Digital.ai VP Platform Strategy Prins points to common symptoms of bottlenecks: "backlog laziness, agenda craziness, pipeline haziness, and delivery bumpiness."
Items lingering in the product backlog, for example, can show differences in priorities compared to assumptions — i.e. a feature considered important in feature planning meetings is not actually important if it lingers in the backlog for months. It is better to either prioritize it, roll it into another release package, or do away with it entirely. Doing otherwise is, technically, being dishonest towards yourselves and your customers.
Similarly, process confusion, or lack of transparency, or stress from juggling several initiatives can show a lack of alignment among teams.
Note that many approval bottlenecks can be resolved through automation and applying analytics to score change items based upon risk. Giving change advisors more contextual information along with automated tools negating the need for full review/approval contributes to agility while eliminating time-consuming manual review.
Bottlenecks related to specific teamwork/approvals can be resolved by shift-left. As suggested above, embrace DevSecOps to implement security during planning and development — rather than in response to a poor security performance review just prior to deployment.
4. Exercise: map IT activities to your company objectives
Conceptualizing lays the groundwork for objective, actionable metrics to be discovered. Knowing this, one of the greatest tools at a DevOps leaders' disposal is an explorer map and an impact outcome map.
An explorer map starts with the determination of the ideal outcome of transformation or fulfillment of VSM initiatives. Once that outcome is identified, it becomes a "North Star" to guide further action. Leaders can trace what they predict are the exact steps needed to get to that target outcome given the current position.
Impact Outcome Map
An impact outcome map works similarly to an explorer map, but it is focused on measurement parameters rather than the look and feel of an accomplished goal. To start impact outcome mapping, list a specific, likely measurable outcome, like "achieve stellar product reputation" or "fuel revenue growth through digital services".
Then, work backward to determine the metrics that best reveal progress towards this goal.
Examples corresponding to both goals above might include:
- Improved product reputation: Volume of perfect review scores, % change in the number of perfect review scores over time
- Leveraging digital to grow revenue: Increased number of digital platform users, increased purchase order size per-customer using a digital platform
Work backward further to identify needed actions taken by customers to drive metric in positive direction:
- The improved volume of perfect review scores: users encounter fewer issues, more users feel compelled to post five-star reviews
- Growing revenue through digital: more people visit the digital platform and use it, customers add more items to their basket prior to checkout
Teams can continue working backward from there to determine the actions needed to drive these key metrics in the positive direction:
- Encourage users to post reviews through email marketing, monitor negative reviews for complaints about product/service performance to guide new changes
- Invest in display advertising to encourage traffic to the digital platform, a/b test cart designs to determine the UI that encourages the most purchase conversions
Both explorer maps and impact outcome maps depict priorities in a way that reveals what metrics might both measure progress towards that goal and feedback for taking action to improve performance towards the goal.
Conceptual work lays the foundation for VSM transformation at the metric level
Monitoring metrics and using them to prompt action is an entire technical discipline that is covered in other areas of our Digital.ai resources. This post is meant specifically to inspire your teams to start conversations that help them not just identify metrics but understand why they matter towards achieving goals.
Unfortunately, because of human nature and the nature of complex business organizations, no amount of unilateral decision-making performed in a vacuum by a transformation team can inspire collaborative buy-in. A lone person using best practices to identify metrics may not necessarily be listened to, or they may fail to inspire the culture change needed to progress towards the target outcome.
A conversation amongst business leaders at multiple levels and crossing through multiple disciplines will tease out the discoveries needed to start moving in the right direction. Input from across the organization also reveals new angles of understanding, such as how an engineering team may unexpectedly have performance goals diametrically opposed to the InfoSec teams. Make sure one team isn't being incentivized to work against another team's most critical goals!
Use the steps outlined above to generate ideas and come to a consensus on what forward progress towards the "north star" goal outcome looks like, how to achieve it measurably, and how to generate feedback to push everyone in alignment towards the right direction. Collaborating on conceptual work encourages buy-in and motivates the culture changes needed to prompt meaningful action.
Learn more about value metric and discover, culture transformation, and determining what steps needed to be taken to push positive progress in our recent webinar: "Metrics That Matter: How to Measure Digital Transformation Success"