One of the things I emphasize in my executive engagements is the need to focus on measuring results rather than expectations, since expectations tend to focus more on operational adherence rather than value delivery.

Couched within this conversation is the idea that Agile is not just about efficiency, it’s also focused on effectiveness (value delivery). After all, what good is a process/method that helps you to do what you do faster/cheaper, but ultimately fails to deliver more value to your end users? Agile, therefore, offers the promise to both decrease costs and to increase revenue.

So, if your emphasis, and maybe your sole reason for choosing to go Agile, is to be more efficient, you will likely see the results you expected; your stuff is going out the door faster. But, if this excludes a focus on increasing revenue/value (i.e. product results), then ultimately you’re really just delaying the inevitable death of your organization. No amount of cost reduction can make up for a lack of revenue production. This idea may suggest that what we produce is more important than how we produce it, but I’ll leave that for another discussion.

Dave Gunther, a colleague at VersionOne, pointed out that process tools don’t really measure product results, and I’m not sure they should ever attempt to do this directly. However, if we view the “efficiency” or process/operational side of this equation as a product (something that attempts to solve a customer problem), then we may be able to find something of value in our process worth measuring.

Consider “pirate metrics”, which offer 5 key metrics to consider for a subscription based business model. Ultimately, these are focused on increasing revenue, which is the final metric. Here they are:

  • Acquisition
  • Activation
  • Retention
  • Referral
  • Revenue

Basically, each of these metrics build on the previous one, so, if you aren’t increasing your acquisitions, then there is no way you will increase your activations, therefore you won’t have increased retention, referrals and ultimately no increased revenue. Or, another way to look at it is that you may have received a referral, which is good, but if you aren’t actively tracking how you came to receive a referral, you don’t have very good chance at recreating that result.

Eric Ries, in his book The Lean Startup, posited that we make guesses on what will actually turn these metric “dials”, and so proposed that instead of investing a bunch of money to fully develop our “assumptions”, we would be better off implementing the scientific method to prove, or rather disprove, them. For example, if you were actively measuring acquisitions, then, hypothetically, every option you choose to implement to your product would affect that measurement. This being true, then we can compare the results of each option to each other to determine which options best get us to our goal of increasing revenue (i.e. value). Essentially, this process is an A/B testing model.

With this model as an example, if we apply it back to the operational/process side of Agile Transformation, what would we consider to be our metrics? Would the revenue equivalent be velocity (I know, don’t compare!)? Would acquisition be equivalent to number of people trained in a boot camp?

What do you think these measurements should be? How do you measure the value of your process options, or do you even measure them at all? Please leave your comments, I’d love to hear what you all think.

Join the Discussion

    • John Krewson

      “Working software is the primary measure of progress”. How about we decide what “working” means for the organization and go from there? Working: no defects. Working: satisfies the users’ needs. Working: satisfies the buyers’ needs. Working: satisfies the accountants’ needs. Each of these definitions imply value and can be associated with a metric.

      • Michael Moore

        Yes, these are great things to measure, but I would contend that they are still about the product, not the process. If the user of the process are the employees/teams who utilize the process, what needs do they have and how can we tell if we are meeting them?

    • Srinath

      I really wonder if you could compare Pirate metrics to Scrum metrics – be it velocity, burndowns, defects or which ever. Would this be covered in the next post ?

    • Jonathan Sickert

      Eric Aries also defines two categories of assumptions that one can build experiments around: the value proposition, and the growth proposition. Here too is a ripe environment for determining what success looks like and development of supporting metrics.

    • Satish Thatte

      Good blog containing several thought-provoking ideas. It is worth recalling what W. Edwards Deming had said “If you can’t describe what you are doing as a process, you don’t know what you’re doing.” My corollary to that statement: Process is very important, but don’t lose sight of your business goals or results. Process has to serve you reach your business goals. Process is not an end in itself, but only a means to an end, which is driven by your business goals or results you are striving to achieve. A bad process may make it very difficult to achieve your business results, but simply following a process diligently does not guarantee business results. You need to focus on them, and work backwards to figure out or fix your process elements that would help you achieve your business results.

      I wrote a blog very much related to this subject where I identified the following business results you may consider striving for.
      1. Understand and satisfy the real requirements of customers
      2. Improve time to market
      3. Reduce release cost (covering development and delivery)
      4. Improve release productivity for software development and delivery process
      5. Improve quality of products shipped
      6. Improve Productivity and Quality concurrently (they should not be traded-off against each other)

      You may see the details at:

    11 + = 13