Home » Schedule » We Really Can Know If Our Project Estimates Are Accurate

The bottom line is that we can know if our project management tool estimating is accurate.  More to the point, estimating in a project can be verifiably accurate.

I recently read the following comment in a conversation about estimating projects:

One nit. We as a community have got to stop using the phrase “mis-estimate”, “incorrect estimate”, and “wrong estimate”.
es-ti-mate:  To calculate *approximately* (the amount, extent, magnitude, position, or value of something).
An estimate that’s not “accurate” is still a valid estimate. An estimate can not [sic] have errors. That’s why it’s called an estimate.

Too often, I’ve seen estimating handled in this way.  The notion is that since it is only an estimate, it doesn’t actually have to be accurate and be without errors.

OK, so what then is the purpose of an estimate?  If we estimate it will take one month to do a task, and it takes three, is that OK?

I don’t think so.

How do we get accurate estimates?  I regularly recommend using past historical performance for getting good estimates on how to do a current project.  This works well because in most environments the People + Processes + Tools do not change significantly from project to project. This results in a regular consistency in performance.

For more see: Get The Project Management Tool Schedule Right

How can we tell how well we estimate?  This is at the crux of the comment above.  Here, the writer didn’t seem to express any belief that an estimate can be accurate in any verifiable way.

So, how do we check how well we are estimating?  Measure it.

Let’s say the last five times we did something it took the following number of weeks:   18, 17,16,19,18 (we have to keep track of how long things are taking to be able to do this).

Looking at our old plans, we see that we estimated the following for the same tasks: 15, 14, 16, 13, 15 (again, we have to keep records to know these values).

Estimate Actual Comment
15 18 Late
14 17 Late
16 16 On-time
13 19 Late
15 18 Late
Avg 14.6 17.6 3.0 weeks late on average
Dev 1.1 1.1 1.1 weeks standard deviation from the average

What we can see in this data is something I typically see in organizations that struggle to deliver projects and products on time.  There is usually a fairly clear and distinct difference between what is being estimated and what generally happens.  Just getting this kind of data into a summary slide helps many folks finally see, or admit to, the pattern.  Too many believe that the estimates are just fine and we would deliver projects on time if folks just didn’t make so many mistakes!

For more on successful change management see:  Why Change Management Is Hard And How To Succeed Anyway

However, just having estimates being different from the actuals is not an indication alone that the estimating is not accurate.

Suppose I come in as a, hopefully, overpaid consultant and work with the team to improve the accuracy of their estimating.  I propose they use the average of the actuals as an estimate and maybe, just maybe, tweak it one way or another a bit if we feel there is something especially significant in this project.

With the next few projects, we get the following results:

Estimate Actual Comment
17 18 Late
19 17 Early
16 16 On-time
17 19 Late
18 18 On-time
Avg 17.4 17.6 0.2 weeks late on average
Dev 1.1 1.1 1.1 weeks standard deviation from the average

As we can see, some of the projects were late but more were early or on-time.  This is also typical of what happens when we improve the estimating.  Estimating isn’t suddenly perfect but clearly we seem to be doing better.

A good indicator is to look at the averages of both the estimates and the actuals.  In the first example, there is clearly a noticeable and consistent difference averaging three weeks late.  In the second example (I purposely made the actuals the same so it is easier to compare), we appear to be doing better at estimating, and consequently delivering on time, with an average of just 0.2 weeks late on average.

More about the potential of simple averages see: Your Average Is Powerful

You may note that in improving the accuracy of the estimate, the estimates “grew” to become closer to the actuals.  This is almost always the case.  Too many organizations want to improve their estimates by getting the “actuals” to be closer to the estimates.   There is nothing wrong with trying to speed up our processes.  However, fixing schedule performance by trying to get real processes to match up with flawed estimates is not the best process improvement approach.  Instead, get an accurate estimate then improve our process and watch the estimates — and the actuals — improve with time.

Some managers would not like hearing that our overall average is now “only” 0.2 weeks late.  This is another dynamic I often see. Once we start to improve accuracy, anything that is not perfect must be a huge problem.  I’ve seen great improvements in estimating and on-time delivery all but discarded because estimates were not perfect and could still results in missed dates.  This is where I get back to reminding folks that an estimate is not perfect. This is where the writer above and I do substantially agree.

I’ve used three fundamental ways to solve this “problem” of imperfect estimates:

  1. Focus on improving the speed and quality of the process to bring down the time and variability and reduce the chances of missing a date.
  2. Add one or two deviations (e.g., 1.1 weeks from the above example) to the average in making the estimate.  This accounts for more of the typical variations seen in the past.
  3. Observe that when given the average to work with, teams will often make improvements that allow them to complete a task faster than the average assumed.  Just about every team we’ve given the “average” amount of time to not only delivered on time, but did so with a remarkable increase in productivity and quality.

For how good schedules improve quality see:  In Project Management 9 + 3 Is Not Equal To 12

Estimating might well be one of the most critical activities any manager will do on a project.  Measuring the difference between our estimates and how we actually do will allow us to know how well we are estimating. We know our project management tool estimating is working fairly well when the average time to complete a task or deliver a product matches closely the average estimate we made for those projects.

How close are your actual project completion dates to your project planned dates?

Thank you for sharing!

10 thoughts on “We Really Can Know If Our Project Estimates Are Accurate

  1. Ravikumar Raja says:

    Hi,

    Yes the experience of lateness in one project would guide all in arriving at the production points for a company. Say you estimate an use case method then we need to know from experience how much an average developer of our company requires in time to complete an usecase(use case points), like wise we need to have all methods of statistics from past to guide us in present and future.

    1. Bruce Benson says:

      The trick, it seems, is to get people to use these kind of techniques. Not everyone needs it, just those organizations who are having a difficult time delivering on time.

      Thanks for the comment.

      Bruce

  2. Bruce Benson says:

    “It is still late!” This was the response from a Fortune 50 senior VP when a brave soul mentioned we had delivered a major product only a week late.

    The conversation ended at this retort from the SVP.

    This is the kind of toughness needed to improve, right? Not!

    Previously, for at least 7 years, we had regularly delivered major products 3 to 6 and even 9 months late. This “one week late” had been a huge improvement.

    The SVP was new (but not new to the company). She apparently didn’t like any improvement that she did not initiate.

    She was gone within a couple of years. Not only did she not make a difference, she had suppressed taking advantage of the biggest improvement we had made in years. All she had to do was say “Great job. We need to do more like that and better and here is what I had in mind ….”

    Success is not always self evident nor self perpetuating. We may have to work hard to keep making progress. The trick is to realize this is fairly normal and we just need to persevere.

    Bruce Benson

  3. Pingback: pliggvote.com
  4. Bruce Benson says:

    I fixed the comments on the 2nd table to match the data (I cut and pasted without updating …).

    Bruce

  5. Dina says:

    Great post! I dared once I use the word ‘guess’ and almost had my head bitten off! A great book that helps detail many different techniques used to improve software estimation is by Steve McConnell, titled “Software Estimation, Demystifying the Black Art”. The title alone shows how difficult it can be.

    One project management tool that can help with the comparison to historical data is LiquidPlanner, which has reports that will show original estimate vs actual, and allow the project manager to see who makes estimates most close to actual, who is way off and by how much. Very handy tools (along with a big pile of other great features!).
    http://www.liquidplanner.com/tour/

    1. Bruce Benson says:

      Dina,

      Yes, too often the “many different techniques” and “black art” notions hide some really straight forward techniques that can work well. I also try to get folks to do it manually, so they understand the technique, before using a tool.

      1. Maturity level 1: compare your last estimate with what actually happened. How close were you? It is amazing how many people have not tried this or lack the data to make this comparison.
      2. Maturity level 2: compare the averages of your estimates to the averages of your actuals (this article). Guess what? You can tell how well you are estimating in general.

      I find that once someone has done the work manually, just did the straight forward calculation and understood the result, then the tools are suddenly useful and they can quickly tell when the tool results don’t make sense.

      Thanks for the notes and the pointers.

      Bruce

Comments are closed.