You Really Can Know If Your Project Estimates Are Accurate

The bottom line is that you can know if your project management tool estimating is accurate. More to the point, estimating in a project can be verifiably accurate.

I recently read the following comment in a conversation about estimating projects:

One nit. We as a community have got to stop using the phrase “mis-estimate”, “incorrect estimate”, and “wrong estimate”.
es-ti-mate: To calculate *approximately* (the amount, extent, magnitude, position, or value of something).
An estimate that’s not “accurate” is still a valid estimate. An estimate can not have errors. That’s why it’s called an estimate.

Too often, I’ve seen estimating handled in this way. The notion is that since it is only an estimate, it doesn’t actually have to be accurate and be without errors.

OK, so what then is the purpose of an estimate? If you estimate it will take one month to do a task, and it takes three, is that OK?

I don’t think so.

How do we get accurate estimates? I regularly recommend using past historical performance for getting good estimates on how to do a current project. This works well because in most environments, the People + Processes + Tools do not change significantly from project to project. This results in a regular consistency in performance (see: Get The Project Management Tool Schedule Right).

How can you tell how well we estimate? This is at the crux of the comment above. Here, the writer didn’t seem to express any belief that an estimate can be accurate in any verifiable way.

So, how do we check how well we are estimating? Measure it.

Let’s say the last five times we did something it took the following number of weeks: 18, 17,16,19,18 (we have to keep track of how long things are taking to be able to do this).

Looking at our old plans, we see that we estimated the following for the same tasks: 15, 14, 16, 13, 15 (again, we have to keep records to know these values).

	Estimate	Actual	Comment
	15	18	Late
	14	17	Late
	16	16	On-time
	13	19	Late
	15	18	Late
Avg	14.6	17.6	3.0 weeks late on average
Dev	1.1	1.1	1.1 weeks standard deviation from the average

What we can see in this data, is something I typically see in organizations that struggle to deliver projects and products on time. There is usually a fairly clear and distinct difference between what is being estimated and what generally happens. Just getting this kind of data into a summary slide helps many folks finally see, or admit to, the pattern. Too many believe that the estimates are just fine and we would deliver projects on time if folks just didn’t make so many mistakes!

However, just having estimates being different from the actuals is not an indication alone that the estimating is not accurate.

Suppose I come in as a, hopefully, overpaid consultant and work with the team to improve the accuracy of their estimating. I propose they use the average of the actuals as an estimate and maybe, just maybe, tweak it one way or another a bit if they feel there is something especially significant in this project.

With the next few projects, we get the following results:

	Estimate	Actual	Comment
	17	18	Late
	19	17	Early
	16	16	On-time
	17	19	Late
	18	18	On-time
Avg	17.4	17.6	0.2 weeks late on average
Dev	1.1	1.1	1.1 weeks standard deviation from the average

As we can see, some of the projects were late, but more were early or on-time. This is also typical of what happens when we improve the estimating. Estimating isn’t suddenly perfect but clearly we seem to be doing better.

A good indicator is to look at the averages of both the estimates and actuals. In the first example, there is clearly a noticeable and consistent difference averaging three weeks late. In the second example (I purposely made the actuals the same so it is easier to compare), we appear to be doing better at estimating, and consequently delivering on time, with an average of just 0.2 weeks late on average.

You may note that in improving the accuracy of the estimate, the estimate “grew” to become closer to the actuals. This is almost always the case. Too many organizations want to improve their estimations by getting the “actuals” to get closer to the estimates. There is nothing wrong with trying to speed up our processes. However, fixing schedule performance by trying to get real processes to match up with flawed estimates is not the best process improvement approach. Instead, get an accurate estimate then improve your process and watch the estimate – and the actuals – improve with time.

Some managers would not like hearing that our overall average is now “only” 0.2 weeks late. This is another dynamic I often see. Once we start to improve accuracy, anything that is not perfect must be a huge problem. I’ve seen great improvements in estimating and on-time delivery all but discarded because estimates were not perfect and could still results in missed dates. This is where I get back to reminding folks that an estimate is not perfect. This is where the writer above and I do substantially agree.

I’ve used three fundamental ways to solve this “problem” of imperfect estimates:

1. Focus on improving the speed and quality of the process, to bring down the time and variability and reduce the chances of missing a date.

2. Add one or two deviations (e.g., 1.1 weeks from the above example) to the average in making the estimate. This accounts for more of the typical variations seen in the past.

3. Observe that when given the average to work with, teams will often make improvements that allow them to complete a task faster than the average assumed. Just about every team we’ve given the “average” amount of time to not only delivered on time, but did so with a remarkable increase in productivity and quality.

Estimating might well be one of the most critical activities any manager will do on a project. Measuring the difference between your estimates and how you actually do, will allow you to know how well you are estimating. You know your project management tool estimating is working fairly well when the average time to complete a task or deliver a product matches closely the average estimate you made for those projects.

Thank you for bookmarking and sharing these articles as this tells me which ones are the most helpful.

Filed Under Schedule | 7 Comments

Tagged With Accuracy, Estimation, Project Management Tools

Comments

7 Responses to “You Really Can Know If Your Project Estimates Are Accurate”

Dina on December 22nd, 2009 9:39 pm
Great post! I dared once I use the word ‘guess’ and almost had my head bitten off! A great book that helps detail many different techniques used to improve software estimation is by Steve McConnell, titled “Software Estimation, Demystifying the Black Art”. The title alone shows how difficult it can be.
One project management tool that can help with the comparison to historical data is LiquidPlanner, which has reports that will show original estimate vs actual, and allow the project manager to see who makes estimates most close to actual, who is way off and by how much. Very handy tools (along with a big pile of other great features!).
http://www.liquidplanner.com/tour/
Bruce Benson on December 22nd, 2009 11:10 pm
Dina,
Yes, too often the “many different techniques” and “black art” notions hide some really straight forward techniques that can work well. I also try to get folks to do it manually, so they understand the technique, before using a tool.
1. Maturity level 1: compare your last estimate with what actually happened. How close were you? It is amazing how many people have not tried this or lack the data to make this comparison.
2. Maturity level 2: compare the averages of your estimates to the averages of your actuals (this article). Guess what? You can tell how well you are estimating in general.
I find that once someone has done the work manually, just did the straight forward calculation and understood the result, then the tools are suddenly useful and they can quickly tell when the tool results don’t make sense.
Thanks for the notes and the pointers.
Bruce
Bruce Benson on December 24th, 2009 9:27 am
I fixed the comments on the 2nd table to match the data (I cut and pasted without updating …).
Bruce
pliggvote.com on January 5th, 2010 10:29 am
You Really Can Know If Your Project Estimates Are Accurate | Project Management Tools That Work…
The bottom line is that you can know if your project management tool estimating is accurate. More to the point, estimating in a project can be verifiably accurate….
Bruce Benson on February 13th, 2010 9:06 pm
“It is still late!” This was the response from a Fortune 50 senior VP when a brave soul mentioned we had delivered a major product only a week late.
The conversation ended at this retort from the SVP.
This is the kind of toughness needed to improve, right? Not!
Previously, for at least 7 years, we had regularly delivered major products 3 to 6 and even 9 months late. This “one week late” had been a huge improvement.
The SVP was new (but not new to the company). She apparently didn’t like any improvement that she did not initiate.
She was gone within a couple of years. Not only did she not make a difference, she had suppressed taking advantage of the biggest improvement we had made in years. All she had to do was say “Great job. We need to do more like that and better and here is what I had in mind ….”
Success is not always self evident nor self perpetuating. We may have to work hard to keep making progress. The trick is to realize this is fairly normal and we just need to persevere.
Bruce Benson
Ravikumar Raja on April 6th, 2010 11:58 am
Hi,
Yes the experience of lateness in one project would guide all in arriving at the production points for a company. Say you estimate an use case method then we need to know from experience how much an average developer of our company requires in time to complete an usecase(use case points), like wise we need to have all methods of statistics from past to guide us in present and future.
Bruce Benson on April 6th, 2010 12:19 pm
The trick, it seems, is to get people to use these kind of techniques. Not everyone needs it, just those organizations who are having a difficult time delivering on time.
Thanks for the comment.
Bruce

Leave a Reply

Project Management Tools | You Really Can Know If Your Project Estimates Are Accurate