When in the course of human events, it becomes necessary for one team to dissolve the bondage to metrics and stretch goals, and to begin to develop software at a sustainable pace, a decent respect to the opinions of their coworkers  requires that they should declare the causes which impel them to the separation.

Now don’t worry, I’m not going to paraphrase the whole Declaration.  But I do think its high time I came clean as to why I seem so anti metric.  I am not actually opposed to metrics themselves, but I feel that we need to think very carefully about what we are measuring and why we are measuring it.  There is a lot of time and energy, and yes money, spent gathering metrics in the average software development shop.  It really doesn’t get any better in an Agile software development shop, which is somewhat surprising.   If you get involved in any of the many fine Agile discussion groups, you will periodically see the question:  “I want to improve my Agile rollout, what metrics will help me do this?”  What usually follows is a mixture of questions and answers, but the result tends to boil down to an argument between folks who call for a series of metrics and targets for said metrics, and folks who question the validity of metrics and what they will accomplish.

The metrics oriented folks tend to be coming from a different discipline, or perhaps a more PMI based knowledge foundation than those of us who are not.  That isn’t a value judgment, but a possible insight into what is different. The classic argument tends to go brannock devicealong the lines of “if it can be measured, it can be improved” (to which Ron Jeffries has provided the best answer.  He said “my shoe size is x, how do you intend to improve it?”).  My worry is twofold.  The first problem is that we are assuming there is always something that must be improved.  Now I’m not saying that Kaizen is a bad idea, but really if we start a project with a set of measurements that are somehow going to show us what we need to do differently, we start with the assumption that what we are about to do won’t be good enough.

My other, far more serious complaint about metrics are the behavior they drive.  They don’t tend to drive the behavior we would expect, and in many ways become very disruptive.  Let’s look at some examples.  When I was first becoming a “lead” programmer, I started reading and studying what I would need to do to become a good leader and manager.  I read about a metric that was at that time considered a good measurement of productivity: source lines of code(SLOC).  This seemed to me to be a ridiculous metric.  I got in a friendly argument with my boss, and then went back to my desk and wrote the following C++ line of code…

std::cout<<“Hello World!” << std:endl;

I then highlighted the line, his ctrl+c then ctrl+v and held it down.  In less than fifteen minutes, I had become the most productive programmer in the company, producing over a thousand lines of code!  Obviously the prudent person doesn’t believe that this is what is called for, but it does drive behavior.  In this case, SLOC drives us to write far more wordy, verbose, and complex code than we normally would.  I feel that the person who can implement logic in 5 lines of code is far more productive than the one who does it in 100 lines.  It also doesn’t allow for refactoring.  If I am measured on new lines of code, how in the world can I justify improving lines that are already there?

Here’s another example much closer to home for the Agile software development world, speedometerespecially scrum.  Lets talk about velocity.  One of the first things a coach asks scrum teams to do is to resist the temptation to compare velocities among teams.  And yet, I am constantly seeing agile managers and scrum masters explaining to “leadership” that they shouldn’t be doing that, and also why velocity isn’t what “it should be”.  I have seen many team talk about how to improve velocity.  The end result is always the same.  They come up with some target velocity, or worse a target rate of acceleration.  The logical conclusion of this is also just as predictable.  First, teams  become paralyzed by the need to make sure their estimates are exactly right, since there will be no wiggle room if they get one wrong.  Next, they start estimating everything much higher, in order to allow themselves wiggle room if something goes wrong.  Neither of these behaviors lets us establish a sustainable pace nor does it live up to the Agile Manifesto value of Individuals and Interactions over Processes and Tools.

So, are metrics all bad? No.  There are things we can count or measure to help us see how we are doing.  There is no single template that will work for everyone, though.  So before deciding what metrics you need for a team or a project, decide what behaviors you are hoping to encourage.  Then ask, “Is there something I can measure that will help encourage that behavior, and also help us understand how we are doing?”  If there is, use it.  But even then consider what unintended consequences you might end up with.  And when you have reached your goal the metric is no longer necessary, so get rid of it.


Join the Discussion

    • Bob Williams

      Metrics are neutral. It’s how we use the metrics that define how useful they are to the organization. I like your thought “before deciding what metrics you need for a team or a project, decide what behaviors you are hoping to encourage.”

      At the end of the day, achieving the metrics is not the ultimate goal. We could achieve all the metrics targeted for a team and not be successful if the software we produce isn’t consumed to make the organization money.

    • Joseph

      I always laugh at the shoe size example, as it obviously focuses on the wrong thing and anyone who would quote it has not thought long enough about the value of the number. Shoes have sizes so that you can get the right ones for your feet. The benefit is to both the wearer (comfort and speed of acquisition) and the cobbler (standardized product and not having to touch people’s feet to measure them…)
      I completely agree that we should be defining ideal behavior first and then using metrics to support, but that is not the end. We also have to establish expectations, reinforce the ideal behavior and address incorrect behavior, aka leadership. Poor performance or even low morale is not a result of having metrics, it is a result of poor leadership.

      • Steve Ropa

        Hi Joseph, thanks for your response.

        Actually the shoe size example fits even better with the historical context in place. The idea of a shoe size(or say a team’s velocity) being something that we can use, but not necessarily “improve” was my point. Some metrics can be useful, but their misapplication is rampant in our industry. I think we agree that leadership is really the goal. I have not ever seen metrics support that leadership, but I have many times seen poor leaders rely on metrics to make up for their shortcomings.

    7 + 2 =