Reject the Tyranny of Metrics
When in the course of human events, it becomes necessary for one team to dissolve the bondage to metrics and stretch goals, and to begin to develop software at a sustainable pace, a decent respect to the opinions of their coworkers requires that they should declare the causes which impel them to the separation.
Now don’t worry, I’m not going to paraphrase the whole Declaration. But I do think its high time I came clean as to why I seem so anti metric. I am not actually opposed to metrics themselves, but I feel that we need to think very carefully about what we are measuring and why we are measuring it. There is a lot of time and energy, and yes money, spent gathering metrics in the average software development shop. It really doesn’t get any better in an Agile software development shop, which is somewhat surprising. If you get involved in any of the many fine Agile discussion groups, you will periodically see the question: “I want to improve my Agile rollout, what metrics will help me do this?” What usually follows is a mixture of questions and answers, but the result tends to boil down to an argument between folks who call for a series of metrics and targets for said metrics, and folks who question the validity of metrics and what they will accomplish.
The metrics oriented folks tend to be coming from a different discipline, or perhaps a more PMI based knowledge foundation than those of us who are not. That isn’t a value judgment, but a possible insight into what is different. The classic argument tends to go along the lines of “if it can be measured, it can be improved” (to which Ron Jeffries has provided the best answer. He said “my shoe size is x, how do you intend to improve it?”). My worry is twofold. The first problem is that we are assuming there is always something that must be improved. Now I’m not saying that Kaizen is a bad idea, but really if we start a project with a set of measurements that are somehow going to show us what we need to do differently, we start with the assumption that what we are about to do won’t be good enough.
My other, far more serious complaint about metrics are the behavior they drive. They don’t tend to drive the behavior we would expect, and in many ways become very disruptive. Let’s look at some examples. When I was first becoming a “lead” programmer, I started reading and studying what I would need to do to become a good leader and manager. I read about a metric that was at that time considered a good measurement of productivity: source lines of code(SLOC). This seemed to me to be a ridiculous metric. I got in a friendly argument with my boss, and then went back to my desk and wrote the following C++ line of code…
std::cout<<“Hello World!” << std:endl;
I then highlighted the line, his ctrl+c then ctrl+v and held it down. In less than fifteen minutes, I had become the most productive programmer in the company, producing over a thousand lines of code! Obviously the prudent person doesn’t believe that this is what is called for, but it does drive behavior. In this case, SLOC drives us to write far more wordy, verbose, and complex code than we normally would. I feel that the person who can implement logic in 5 lines of code is far more productive than the one who does it in 100 lines. It also doesn’t allow for refactoring. If I am measured on new lines of code, how in the world can I justify improving lines that are already there?
Here’s another example much closer to home for the Agile software development world, especially scrum. Lets talk about velocity. One of the first things a coach asks scrum teams to do is to resist the temptation to compare velocities among teams. And yet, I am constantly seeing agile managers and scrum masters explaining to “leadership” that they shouldn’t be doing that, and also why velocity isn’t what “it should be”. I have seen many team talk about how to improve velocity. The end result is always the same. They come up with some target velocity, or worse a target rate of acceleration. The logical conclusion of this is also just as predictable. First, teams become paralyzed by the need to make sure their estimates are exactly right, since there will be no wiggle room if they get one wrong. Next, they start estimating everything much higher, in order to allow themselves wiggle room if something goes wrong. Neither of these behaviors lets us establish a sustainable pace nor does it live up to the Agile Manifesto value of Individuals and Interactions over Processes and Tools.
So, are metrics all bad? No. There are things we can count or measure to help us see how we are doing. There is no single template that will work for everyone, though. So before deciding what metrics you need for a team or a project, decide what behaviors you are hoping to encourage. Then ask, “Is there something I can measure that will help encourage that behavior, and also help us understand how we are doing?” If there is, use it. But even then consider what unintended consequences you might end up with. And when you have reached your goal the metric is no longer necessary, so get rid of it.