“I’m always on the lookout for ideas concerning software estimation, even as I ponder my own eternal mantra: ‘Estimates are always wrong.’ … [A]gile methods avoid the extremes of compression and relaxation. Relaxation is avoided by breaking traditional slow-to-deliver projects into small agile pieces, each easily delivered within the related market windows. Working with these pieces also serves to avoid compression, since the same number of people can deliver the smaller agile pieces more quickly.” Geoffrey Lowney, letters to the editor, Communications of the ACM, August 2011.

Geoffry’s comment to an article in the CACM caught my attention as it matches well with the comments I’ve heard in organizations that are struggling to deliver projects on time. The first notion is that estimates are just that, estimates, and hence are by definition wrong. The second is that there is some method “X” — usually several Xs being hotly debated — that if we use it, it will solve our problems.

There is nothing inherently wrong with either of these notions. Both have a basis in reality because for many organizations the first notion of estimates always being wrong is a daily reality. The second notion that there is some method “X” that will help us is confirmed by these same organizations finally being able to deliver projects on time with good quality.

Where I’ve seen things go wrong is where these oversimplified notions become entrenched in the organization. We no longer really try to do a good estimate or improve upon it, because they will “always be wrong” anyway. We just “run the numbers” produce the estimate, react to the push back that it must be faster than that, and then settle upon an estimate that is organizationally acceptable. “Smart” project managers first figure out what an acceptable schedule would be and then adjust their inputs to the estimation process so that it comes out as expected. Breaking this “bad estimation” culture with a techniques such as “brutal honesty” is a key step we’ve used to finally get estimates to be realistic and accurate (i.e., we deliver when we say we will deliver).

The second notion of a methodology “X” being our saving solution is illustrated by Geoffrey’s description of how Agile avoids compression and relaxation. Because we use small pieces we can “easily” deliver in the market window, he says. The hitch here, as we’ve seen this before, is the hint is that “small” means things suddenly work out. As seductive as this sounds, and that is the key — it sounds so reasonable and intuitively obvious — it rarely works this easily. I recall similar arguments that incremental development would fix waterfall projects (another “make it small and it will now suddenly work” approach). In one organization the inspired solution was to have the programmers do all the testing that the independent test team was doing. If the programmers had to do it, it was argued, they would find all the errors the test teams found and hence their code would no longer ever have these errors (it didn’t work out that way). The most humorous example was an early course I reviewed on setting up web sites. The course material said that if an organization were to just set up an internal web site, then just about all company communications problems would … go away.

Estimates don’t have to always be wrong and solutions to our problems are not always the latest methodology fad. Fully understanding and applying both a brutally honest estimating technique and a disciplined and practiced project management methodology will ensure our projects are successful. Just beware of oversimplifying, the problems to be fixed and their solutions, as sometimes the oversimplification becomes part of the problem.

What real problems are oversimplified in your project or organization?