How to Choose the Right Agile Metrics (Part I)

Most organizations are finding value in agile methodology because it offers flexibility and the ability to change course as needed. That is particularly helpful in IT since both business needs and technology are constantly evolving. The key to agile success is using performance-based metrics to measure quality and productivity. But teams must be careful to ensure that the metrics they choose provide benefits that outweigh the overhead of creating them.

How can you know which metrics to track to ensure success? And how should teams contextualize metrics to produce meaningful and actionable data? When is a metric not right for you?

In this blog, we will explore these questions by analyzing three metrics. We will cover the remaining three metrics in our next document as part two of this series

1. Task (or Story) Completion Ratio:

This simply refers to the ratio of committed vs. completed tasks. E.g., if a team plans 100 tasks in a sprint and completes 80, they have a completion ratio of 80%. Sprint tasks are supposed to be small, manageable and, above all, efficient. Some teams measure story completion instead, but both methods share the same aim: to get a sense of how well the team is forecasting its capacity when planning sprints.

The Benefits

If there are gaps between expectations and results, this ratio could indicate problems with the team’s capacity forecasts or task assignment methods. The goal of a sprint should be to produce a consumable product, especially if the team is moving towards Continuous Integration (CI) and Continuous Delivery (CD). To facilitate this, the workload for each sprint should remain reasonably constant and stable.

How Not to Use It

This metric should be used to attack the problem, not the personnel, and should be published after the team has reached a consensus on how to improve task estimation for the next sprint.

Challenges and Other Considerations

One common challenge is getting the team to reliably update their task information into the agile tool on a regular basis. All too often, teams enter their data towards the beginning and the end, but updates are light in between. Another example is failing to link tasks with corresponding defects.

Taking team optimism into account during capacity planning is another challenge in benchmarking this metric. Often, teams overestimate their capacity and commit to more items than they can deliver. Lacking full knowledge of the requested functionality is one of the challenges in forecasting capacity correctly.

2. Time blocked for a work item:

This metric describes the impact and duration of blockers (also referred to as impediments) on work items. It ties in with the previously-mentioned task completion ratio.

The Benefits

Most development tools feature time blocker tracking for each work item. At the end of a sprint, simply add up all the time that a task spent in a blocked state. Repeat that for all tasks and divide by the total number of tasks to get an average time blocked for each task.

The blocked time data should be analyzed over a period to identify irregularities in the system that may require corrective measures. Examples of issues that may cause irregularities include unplanned vacations, system issues like memory issue, hardware downtime, support unavailability, etc. The data from this metric should be analyzed to differentiate one-off blocking patterns from systemic points of failure.

How Not to Use It

This metric should only serve to keep track of time elapsed for tasks and to measure the delays caused by blockers, which are often factors outside the team’s control such as approvals by other teams.  The goal is to improve accountability and performance, not punish purported inefficiencies.

Challenges and Other Considerations

The main challenge is to reinforce regular data entry from all team members. Disciplined adherence to this standard can be encouraged through emphasis and education. Another challenge is the proper usage of the tools. Tool customization options may help encourage team members to consistently enter data.

3. Team velocity

Team velocity refers to the work product completed during a single sprint. To calculate this data, simply add up all the completed story points at the end of the sprint, with the definition of “complete” set by the team.

The Benefits

The data collected from this metric should be used to set a workload benchmark for the team. While it’s normal for a team’s velocity to oscillate from sprint to sprint, ideally a team’s velocity should increase overtime and stabilize at an efficient rate. One of the primary benefits is clear feedback on whether a certain adjustment or action is improving the team’s productivity or hurting it. Another benefit is improved cohesion between team members.

Challenges and Other Considerations

Only consider completed stories or items, not partially-completed or incomplete stories. Also, it is essential to estimate the entire set of stories before the project starts, or early on, and in a consistent manner.

There can be a change in team velocity due to various factors, such as change in team members, temporary assignments, new business knowledge requirements, unplanned absences, etc. Careful consideration must be exercised when deploying a team to another area of business, as their velocity may change considerably due to a shift in the nature of their work.

Learn how to realize the promise of Agile and DevOps by continuously measuring and improving your adoption of best practices in this webinar.

Watch the Webinar