Agile Metrics – Measuring What Actually Matters
Do you really know how your agile teams are doing? As organizations transform to agile and stand/up multiple teams it becomes almost impossible to see how healthy all those teams are in a consistent way. This is an important question not just for leaders, but for the teams themselves to understand the areas where they need to improve. Here’s a typical view of what happens today to answer this question:
- Every agile coach or ScrumMaster creates their own survey or evaluation of team health and maturity.
- We leverage excel sheets to analyze the data which could get complicated and take weeks/months to analyze.
- There is no consistent way of measuring health across many teams.
- Agile coaches are stretched too thin with no clear way to prioritize which teams need their help the most and which teams can use self/manage.
- Leaders and executives don’t have overall understanding of the health of the organization and where they can help. We tell them ‘agile is working! We FEEL we’re making big progress’ and then only share hard metrics to prove it.
When we talk about the health of teams, we shouldn’t just focus on the hard metrics and agile processes alone. This can lead to gaming of the data. The metrics that give us ‘predictive’ indicators actually come from the cultural and people side. This is called ‘mode 2’ of the bimodal movement according to Gartner. Susan Courtney, CIO of BCBS NE, actually shared that “80% of their enterprise agile transformation was about the cultural side and 20% about the agile and process side”. So… We started a couple of years ago digging deeper into this topic of team and organizational health measurement. Our objectives were simple:
- Have a holistic measurement of team health that is consistent across all teams so that we can identify common patterns and address them.
- Engage a team in a fun strategic deep/dive retrospective that gives them immediate value. We’ve learned this needs to be a facilitated conversation that allows them to analyze the results of their assessment in real/time so they can build an actionable growth plan.
- We wanted the results to appear in ONE engaging visual. We were tired of analyzing several graphs to make sense of the data. Hence the birth of the TeamHealth Radar.
- This needed to happen on a cadence. Similar to how teams do Release Planning every quarter or every Program Increment (PI) (if you’re using Scaled Agile Framework® or SAFe®), we wanted them to inspect and adapt quarterly and then work the growth items during each spring retrospective. Why? So growth is continuous and measurable.
What does business agility and organization health look like? Let’s go 5,000 feet above before we start digging into TeamHealth. As we’ve worked with many companies attempting to transform themselves and looked at the common objectives for their transformation, four key pillars started to emerge: Clarity: “Do we know who we are and why we exist? Do we have agreement on our core values and how we should behave? Do we know what’s the most important thing right now at the team, program, and portfolio levels?” Focus: “Are we able to stay focused until we get something done or do we continue to run after shining objects?” “Do we allow our teams to finish what they started before adding new ideas to their backlog?” Most of the lack of focus actually comes from lack of enterprise adoption of agile planning and prioritization at the portfolio level. That’s for a different future blog! Predictable Execution: “Do we all know how to execute work in a predictable way, whether is Kanban, Scrum, or Lean. Do we have a process for it where we all know what it means to be a product owner, how often do we plan our iterations, and what does velocity means for us?” Healthy Culture: In the middle of that triangle are the healthy and happy people who enjoy what they are doing. This comes from their leaders shifting from command and control to servant leadership, and their teams learning strong collaboration, facilitation, and teamwork skills. What We’ve Learned About Team Health We’ve spent a few years now learning and improving our understanding of what makes an agile team (or any team) healthy through building AgilityHealth. The heart of this discovery has been the TeamHealth Radar below in addition to the format of the facilitated quarterly retrospective. This radar provides ‘qualitative’ measures that are critical to understanding how the team is doing and where they can improve. I’ll tackle including ‘quantitative’ later in future blog. Clarity A healthy team should have clarity on their vision and purpose (why they exist) and their measure for success. They should have clarity on their plan; short term, mid/term, and long term. They should have clarity on their roles; what is expected of me individually and what is expected of others on my team? Another critical skill is becoming generalizing specialists. As a team member, am I open to learning new skills beyond my specialty so I can help my team when needed? Performance Performance is measured in two ways: confidence (the gut/check) and measurements. Confidence, starting with the product owner, we ask: “As a product owner are you confident and satisfied that this team has the tools, the skills, and the desire to meet your current goals?” We ask the same question to the team members. We also engage the stakeholders outside of the team (users, sponsor, managers, other teams) by asking them for their confidence in the team AND their satisfaction using the NPS popular question ‘How likely are you to recommend working with our team to others?’ Measurement, we’ve gathered the top five drivers for companies adopting agile (see VersionOne State of Agile Report). These are: predictable velocity, time to market, value delivered, quality, and response to change. Leadership A strong and healthy leadership team has a direct impact on the health of the overall health of an agile team. To assess their health, we focus on these roles: team facilitator (ScrumMaster), product owner, technical lead, and the functional manager of the team. We’ve found interesting relationships emerge. For example, there is a correlation between clarity (as an outcome), and leadership (as an input). This means, as an example, if you want the team to have clarity on vision, plans, and roles, they should have a healthy team facilitator and product owner. We also believe it is important to assess the functional manager of the team due to the positive or negative influence their leadership can have on the team. We focus on servant leadership, people development, and process improvement. This is important in order to nudge managers to shift their focus on growing individuals, and improving their team process rather than task management and fire fighting. Culture Probably the most critical aspect of the health of a team is that layer below the surface. Part of the power of having this will be a facilitated TeamHealth retrospective each quarter, is it provides the team an opportunity to to dig deeper into this layer and open up conversations they usually don’t have during regular iteration retrospectives. We measure cultural health by having each person rate the following ‘happiness’ statement on a 10/point scale “I enjoy working with this team”. We then dig into how well the team collaborates, do they trust and respect each other, are they allowed to be creative (and do they actually do it). Finally, do they hold each other accountable or are they still waiting for a boss to do that? Foundation Healthy agile teams have a strong foundation. This is made up of basic principles of agility such as sustainable pace (not burning out), self/organization (empowering the teams to make decisions), technical excellence (which now has its own Technical Health radar), having the proper planning and estimating cadences, and facilitating effective meetings. The team structure can also have a big impact on health and performance. We assess if the team has the right size and skills, and are they allocated and stable (reduce multi/tasking and pulling people out of the team). Finally is their workspace environment, virtual or co/located, and setup for collaboration? Summary and Final Thoughts As I’ve worked with many companies through their agile transformation it has become clear to me and to the leaders of these organizations that measurement is no longer a nice/to/have, but a must/have. As you scale agile and stand/up tens or hundreds of agile teams, the visibility and transparency of how they’re doing is imperative to your future success. Seeing patterns across multiple teams and building much more targeted growth at the team, program, and portfolio/LOB levels takes your transformation to the next level of maturity. I hope you’ll share our vision and passion for leveraging the right set of metrics to enable business agility. Scaled Agile Framework and SAFe are registered trademarks of Scaled Agile, Inc.