This post is from the CollabNet VersionOne blog and has not been updated since the original publish date.
Why I don’t like the metric system
I like to get involved in the various agile development discussion groups. These groups tend to provide a ton of insight into how to make extreme programming, scrum, or any other flavor of agile development that much more than “by the book”. Lately, there have been quite a few comments posted regarding metrics. The questions are usually something along the lines of “how can I use agile metrics to prove that agile is helping my software development project?”
I have a problem with metrics. Metrics seem to be measuring something because you can measure it. When I see or hear someone asking for a set of metrics that should be defined prior to the actual development project, I get worried. Are we moving away from the agile manifesto value of working software? Are we letting ourselves get dragged into the same trap that the waterfall crowd fell in?
Lets explore some of the reasons one might to want metrics. The most commonly stated reason is to be able to measure progress, so that we might improve. What exactly we are going to improve I don’t know, but it sure is going to get better. What can we measure in an agile environment that might help us improve?
1. Velocity – So many teams see this as something they can improve on. “We had a velocity over the last 4 iterations of 25 points worth of stories per iteration. Surely if we just try a little harder, we can to 30.” Making this statement implies you don’t believe we tried hard enough this time. If a team member makes this statement, they are generally referring to someone else on the team. If a product owner or team leader of some type makes this statement, they are directly accusing the team of sandbagging. So measuring velocity is fraught at best. I look at velocity as a way to get a feel for the capacity of an upcoming iteration. I hope, over time, that i will start to see some predictability in my velocity. I never ever ever look for a way to increase velocity.
2. Estimate Accuracy – I used to see this one a lot in the scrum software development discussions. The desire to improve estimation accuracy just seems to be built into the project management psyche. I see where the desire comes from, but it really only applies to a waterfall methodology. We want to be able to estimate well so that our software project will stay on track. If we are good at guessing how long something will take, we will be able to predict better how long the whole project will take. But in Agile, we are starting off by saying “we recognize that we can’t estimate well.” We are going to accept that fact, and we are going to instead work on ways to create great software, and instill predictability, with that fact in mind. Short iterations mean we don’t have to try to predict so far into the future. Small user stories mean we don’t have to try to estimate something so complex as to be virtually incomprehensible.
So I’m not crazy about metrics. They tend to drive behavior that is around making the numbers look good, not making the software look good. But maybe I’m being too “purist”. Are there metrics out there that *can* help us be better at being agile? Are there metrics that folks have run across that I didn’t mention that have gotten in the way of their agile implementations?