Agile Metrics: Measuring Process Value
One of the things I emphasize in my executive engagements is the need to focus on measuring results rather than expectations, since expectations tend to focus more on operational adherence rather than value delivery.
Couched within this conversation is the idea that Agile is not just about efficiency, it’s also focused on effectiveness (value delivery). After all, what good is a process/method that helps you to do what you do faster/cheaper, but ultimately fails to deliver more value to your end users? Agile, therefore, offers the promise to both decrease costs and to increase revenue.
So, if your emphasis, and maybe your sole reason for choosing to go Agile, is to be more efficient, you will likely see the results you expected; your stuff is going out the door faster. But, if this excludes a focus on increasing revenue/value (i.e. product results), then ultimately you’re really just delaying the inevitable death of your organization. No amount of cost reduction can make up for a lack of revenue production. This idea may suggest that what we produce is more important than how we produce it, but I’ll leave that for another discussion.
Dave Gunther, a colleague at VersionOne, pointed out that process tools don’t really measure product results, and I’m not sure they should ever attempt to do this directly. However, if we view the “efficiency” or process/operational side of this equation as a product (something that attempts to solve a customer problem), then we may be able to find something of value in our process worth measuring.
Consider “pirate metrics”, which offer 5 key metrics to consider for a subscription based business model. Ultimately, these are focused on increasing revenue, which is the final metric. Here they are:
Basically, each of these metrics build on the previous one, so, if you aren’t increasing your acquisitions, then there is no way you will increase your activations, therefore you won’t have increased retention, referrals and ultimately no increased revenue. Or, another way to look at it is that you may have received a referral, which is good, but if you aren’t actively tracking how you came to receive a referral, you don’t have very good chance at recreating that result.
Eric Ries, in his book The Lean Startup, posited that we make guesses on what will actually turn these metric “dials”, and so proposed that instead of investing a bunch of money to fully develop our “assumptions”, we would be better off implementing the scientific method to prove, or rather disprove, them. For example, if you were actively measuring acquisitions, then, hypothetically, every option you choose to implement to your product would affect that measurement. This being true, then we can compare the results of each option to each other to determine which options best get us to our goal of increasing revenue (i.e. value). Essentially, this process is an A/B testing model.
With this model as an example, if we apply it back to the operational/process side of Agile Transformation, what would we consider to be our metrics? Would the revenue equivalent be velocity (I know, don’t compare!)? Would acquisition be equivalent to number of people trained in a boot camp?
What do you think these measurements should be? How do you measure the value of your process options, or do you even measure them at all? Please leave your comments, I’d love to hear what you all think.