The bottom line is that we can know if our project management tool estimating is accurate.  More to the point, estimating in a project can be verifiably accurate.

I recently read the following comment in a conversation about estimating projects:

One nit. We as a community have got to stop using the phrase “mis-estimate”, “incorrect estimate”, and “wrong estimate”.
es-ti-mate:  To calculate *approximately* (the amount, extent, magnitude, position, or value of something).
An estimate that’s not “accurate” is still a valid estimate. An estimate can not [sic] have errors. That’s why it’s called an estimate.

Too often, I’ve seen estimating handled in this way.  The notion is that since it is only an estimate, it doesn’t actually have to be accurate and be without errors.

OK, so what then is the purpose of an estimate?  If we estimate it will take one month to do a task, and it takes three, is that OK?

I don’t think so.

How do we get accurate estimates?  I regularly recommend using past historical performance for getting good estimates on how to do a current project.  This works well because in most environments the People + Processes + Tools do not change significantly from project to project. This results in a regular consistency in performance.

For more see: Get The Project Management Tool Schedule Right

How can we tell how well we estimate?  This is at the crux of the comment above.  Here, the writer didn’t seem to express any belief that an estimate can be accurate in any verifiable way.

So, how do we check how well we are estimating?  Measure it.

Let’s say the last five times we did something it took the following number of weeks:   18, 17,16,19,18 (we have to keep track of how long things are taking to be able to do this).

Looking at our old plans, we see that we estimated the following for the same tasks: 15, 14, 16, 13, 15 (again, we have to keep records to know these values).

	Estimate	Actual	Comment
	15	18	Late
	14	17	Late
	16	16	On-time
	13	19	Late
	15	18	Late
Avg	14.6	17.6	3.0 weeks late on average
Dev	1.1	1.1	1.1 weeks standard deviation from the average

What we can see in this data is something I typically see in organizations that struggle to deliver projects and products on time.  There is usually a fairly clear and distinct difference between what is being estimated and what generally happens.  Just getting this kind of data into a summary slide helps many folks finally see, or admit to, the pattern.  Too many believe that the estimates are just fine and we would deliver projects on time if folks just didn’t make so many mistakes!

For more on successful change management see:  Why Change Management Is Hard And How To Succeed Anyway

However, just having estimates being different from the actuals is not an indication alone that the estimating is not accurate.

Suppose I come in as a, hopefully, overpaid consultant and work with the team to improve the accuracy of their estimating.  I propose they use the average of the actuals as an estimate and maybe, just maybe, tweak it one way or another a bit if we feel there is something especially significant in this project.

With the next few projects, we get the following results:

	Estimate	Actual	Comment
	17	18	Late
	19	17	Early
	16	16	On-time
	17	19	Late
	18	18	On-time
Avg	17.4	17.6	0.2 weeks late on average
Dev	1.1	1.1	1.1 weeks standard deviation from the average

As we can see, some of the projects were late but more were early or on-time.  This is also typical of what happens when we improve the estimating.  Estimating isn’t suddenly perfect but clearly we seem to be doing better.

A good indicator is to look at the averages of both the estimates and the actuals.  In the first example, there is clearly a noticeable and consistent difference averaging three weeks late.  In the second example (I purposely made the actuals the same so it is easier to compare), we appear to be doing better at estimating, and consequently delivering on time, with an average of just 0.2 weeks late on average.

More about the potential of simple averages see: Your Average Is Powerful

You may note that in improving the accuracy of the estimate, the estimates “grew” to become closer to the actuals.  This is almost always the case.  Too many organizations want to improve their estimates by getting the “actuals” to be closer to the estimates.   There is nothing wrong with trying to speed up our processes.  However, fixing schedule performance by trying to get real processes to match up with flawed estimates is not the best process improvement approach.  Instead, get an accurate estimate then improve our process and watch the estimates — and the actuals — improve with time.

Some managers would not like hearing that our overall average is now “only” 0.2 weeks late.  This is another dynamic I often see. Once we start to improve accuracy, anything that is not perfect must be a huge problem.  I’ve seen great improvements in estimating and on-time delivery all but discarded because estimates were not perfect and could still results in missed dates.  This is where I get back to reminding folks that an estimate is not perfect. This is where the writer above and I do substantially agree.

I’ve used three fundamental ways to solve this “problem” of imperfect estimates:

1. Focus on improving the speed and quality of the process to bring down the time and variability and reduce the chances of missing a date.
2. Add one or two deviations (e.g., 1.1 weeks from the above example) to the average in making the estimate.  This accounts for more of the typical variations seen in the past.
3. Observe that when given the average to work with, teams will often make improvements that allow them to complete a task faster than the average assumed.  Just about every team we’ve given the “average” amount of time to not only delivered on time, but did so with a remarkable increase in productivity and quality.

For how good schedules improve quality see:  In Project Management 9 + 3 Is Not Equal To 12

Estimating might well be one of the most critical activities any manager will do on a project.  Measuring the difference between our estimates and how we actually do will allow us to know how well we are estimating. We know our project management tool estimating is working fairly well when the average time to complete a task or deliver a product matches closely the average estimate we made for those projects.

How close are your actual project completion dates to your project planned dates?