Blah.Blah.Blog

So what then do people think of using these two theoretical KPIs for an Agile development team:

1. Time, in the form of variance between original estimates versus completed actuals
   
2. Code quality in the form of the number of bugs produced by the team

"What's wrong with this picture?", I hear you say:

• The first criterium implies that the project values "estimate accuracy over code quality". After all, if you're stuck finishing a story in time and are under pressure not to exceed estimates, then you're not going to be developing at an optimal level and code quality will suffer. This drop in quality might be merely a side-effect of the project culture, but it could also be a conscious decision to not test as thoroughly as you want to, thereby saving some previous hours. Of course, we know that "saved time" is spent elsewhere, but we'll get to that later...

• Unfortunately, the quality debt we've built to ensure we never exceed our estimates will have to be demanding payment in the form of the number of bugs escaping the clutches of the team and into the wild. But wait, we're also being measured by the number of bugs we produce! Whatever is the solution?

So what other parameters do we have to play with when quality and time have been locked down in this fashion? Well, the two remaining project variables thus far unconstrained are cost and scope. As it turns out, neither option is particularly pleasant:

• Dropping scope just to ensure accurate estimates, while possible, doesn't seem like a good tradeoff. Do we really want to deliver less business value just to ensure we're able to estimate stories accurately against thin requirements? Is maximizing estimate accuracy really more important than maximizing business value?

  Moreover, the sort of intellect that would form KPI's as stated above is probably unlikely to accept such cavalier scope reduction.

• Unfortunately, a more likely solution lies across the cost axis. Specifically, the development team is likely to artificially pad their estimates to ensure they have enough time to build in sufficient quality to satisfy both their internal quality barometer and prevent large numbers of bugs running free.
  

I seriously doubt most PMs run through these scenarios in their heads before deciding how to measure a team's performance. The result of such negligence is a development team that is coerced into creating a communication barrier in the form of inflated estimates when they're working within an environment where transparency and honesty in such things should be valued above anything else.

Just like when you first learn basic physics in school, project physics using the four control variables can take a little getting used to, but the underlying principles are sound and proven over the ages. The more people who understand the interaction between these variables, the better.