“Establish up front how you’re going to know if the [agile] adoption was successful. It could be productivity of the team, less software bugs or more customer satisfaction ….” Software Development Times, April 2012.

What is wrong with this picture? If the only time we want to know how well we are doing is when we make a change, then how do we know — objectively — we even need to make a change?

If we don’t have some kind of measures in place (e.g., productivity, quality, satisfaction, etc.) then that is the first challenge to overcome. Once we know in some objective form how well we are doing, then we can make changes and see — again objectively — what our impact is.

Some simple ways I’ve measured my own projects, included computing the average time to fix a reported defect (as well as the trend of the average time), comparing my milestone achievements (i.e., how long) to milestone achievements in past similar projects, average time to define a requirement, average time to deliver a new project, etc.

Since I had these kind of measures, then when we changed things, I could tell if it made any difference (see meeting madness for an example). It was amazing how few of our improvement efforts had any impact on any of these measures (see kick the habit for more examples).

One day the development VP announced that his product build team had significantly speeded up the build process (an often complained about problem). Now that they’ve speeded up their part of the process, the development team (which also worked for him) needed to now speed up their part of the process. Since I tracked this kind of performance (the speed a defect or feature went through our development process), I was intrigued at this claim and was wondering why I had not noticed it.

I dug into the data and quickly discovered a rather humorous pattern. The build team did indeed now have a speedier process. The first humorous part was that the development team had also sped up in that same time and in fact now comprised an even smaller percentage of the process duration than they had in the past. So build had improved but development had improved even more. None of this was visible to the VP (he later came to rely on my data) nor to the build team manager.

I sent the build team manager my findings. Often when I had done this kind of thing in the past, sent people data that put doubt to their claim, many people were quite unhappy. This build team manager not only appreciated the feedback (and that I had not mentioned it to anyone else) but also asked me to regularly update him on these trends.

The second, equally humorous, part of this insight, was that neither team had done anything special. In looking at past projects, it was just normal for both teams to speed up during this part of a project. It was not that they could not improve, it was just that we were claiming “improvement” when in fact it was just the normal pattern in the project performance (I’ve often seen folks claim improvements or problems based upon the random fluctuation of process performance).

Having simple measurements in place can make a huge difference in how well we perform and equally how well we improve. If the only time we want to measure our performance, in an objective measured way, is when we are pondering the need to change or improve, then we are probably missing a major and critical part of our everyday project management toolset. If we are doing our job, we should not need to set up or decide on measurements to determine if our improvement efforts are making a difference. They will already be in place.

What measures do you have in place on your project that lets you know how well you are performing and when you’ve improved your performance?