Three Ways Project Management Estimates Fail And How To Avoid Them
Estimating the project is one of the greatest challenges, with or without fancy project management tools, that a manager will face. Here are three areas where we’ve seen estimates fail and how we’ve avoided doing the same.
In You Really Can Know If Your Project Management Estimates Are Accurate, I give an example of how using just averages may be able to improve your estimating. Improving your estimating means that what you estimate is very close to what happens. Many folks find this very difficult to do. I’ve observed that it is often difficult for three factors:
1. Predicting the future
2. Remembering the past
3. Keeping it all organized
Let us look at each area in more detail.
Predicting The Future
This is what estimating is all about. Predicting what it is going to take to do something. If we have never done it before then this can be a real stressful challenge. While I’ve significant experience as a software development manager, in a new environment for example all my experience may not do me a lot of good. In one case I was asked to estimate (actually, to confirm an estimate) of how long it would take to develop a whole new customer care system. I was brand new and had no idea. I could have walked through all the details and then used my experience to tweak it. I knew from experience that it would be truly a random walk of logic. Instead, I asked all the “old timers” how long it took to initially develop the current customer care system. Their answer? About two years. Do you know what my estimate was? Yes, two years. This of course was twice the current accepted estimate. We never did write a new customer care system. We just bought another company who had developed something we liked. It took us about a year that way.
While unexciting appearing, using past performance to predict our future performance often works better than many conventionally detailed approaches. (See Get The Schedule Right! for more examples.) This is especially true for organizations which regularly have multiple projects in a year (e.g., software, consumer electronics).
Remembering The Past
The one insight I’ve always leaned on is that most project mistakes are errors of omission. If we remember what we did, we will often be able to do it again and not have to scramble to recover from missing something critical. The trick in any large project is to try and remember everything that was done in the past and to learn from it. This is very hard, and the source of most of what goes wrong.
To help reduce the errors of omission, I would find myself digging through the past records of completed projects. I would look at every project I could get my hands on, even if it was not that similar to my project. Records such as meeting minutes were often much more useful than project plans. The meeting minutes generally showed what came up and what happened over time. Project plans (Microsoft Project, budget, etc.) too often were the “last” version of the project and didn’t show how the project had changed over time. This “changed over time” view of a project provides incredible insight into the real dynamics of most projects.
Another part of what goes wrong is we often remember things optimistically or, just as often, assume we will not make all the mistakes that were made in the past. I’ve never found that to be the case. Even in my example above, asking the “old timers” how long things took, I specifically didn’t ask them what needed to be done. I just asked enough of them to get a consensus average on how long it took. I knew it was easier to remember when they started and when they ended then it would be to recall all the details between those two dates. Many also could remember that it took about twice as long as they had originally planned. The profound insight here is that the duration, two years in this case, provides a time-frame that had a high probability of encompassing all the time needed. Why? Because a real project, to do what we needed to do, was truly accomplished in that time frame. It was doable. We did it in the past. Assuming it will be done in half the time because we’ll avoid all the problems we had before is too often just wishful thinking.
Remembering the past does not necessarily mean knowing all the details. Using the overall duration of an effort or of major milestones completed in the recent past, helps to ensure our team will have the needed time to do it right in the current project.
Keeping It All Organized
How do we normally plan a project? We have multiple meetings and research efforts to work out all the details. From these details we estimate how long something will take and how much it will cost. From there we string together all our details into a project plan. Using the project plan, we control and track the effort and take remedial actions when things don’t go as planned. Sounds simple. Easy to do.
What are we really doing when we plan? We go to our experts, who hopefully know what they are doing, and get certain key milestones from them. These key milestones represent, ideally, a value added point in the project. A point where something useful and hopefully demonstrable, has been accomplished. How do the experts do their part of the project? Each of their departments usually have an established process for planning and doing the work. Rarely does a project plan ever capture the details of how these experts and their departments do the job. To the plan, it is just a black box. It shows a duration for a task (or maybe a range of soonest to latest) and at the end of the task, something useful to others has been completed. The key point is that we must rely on our experts to actually know what needs to be done. The plan can never have all these details.
So how do we help our experts and ourselves keep it all organized? Many project managers (and managers in general) “help” by persistently asking the experts if they have done their job today. Useful? No. Instead we should help by ensuring the teams have the resources (time, money, information) they need to get their job done. While this seems obvious, I’ve observed that many struggling organizations have a tendency to substitute additional oversight (meetings, reports, metrics) in lieu of providing the needed decisions, resources and removing impediments. The curious notion that is often employed is that if we can just watch the experts closely, we can tell them what to do so they don’t need as much resources and can avoid problems. Gee. Maybe we got the wrong people designated as the experts? I’ve never seen more oversight fix an organization that regularly struggled with delivering projects as planned.
Effective project management estimating can be a challenge. Most failed projects, in my experience, have failed due to poor estimating (and managing to those estimates) and not due to poor execution by our teams. Getting these three areas right of predicting the future, remembering the past and keeping it all organized have helped us get projects and organizations to consistently deliver as planned.
16 thoughts on “Three Ways Project Management Estimates Fail And How To Avoid Them”
Comments are closed.
Bruce, I always enjoy reading your blog posts!
To the discussion re: who should make the estimate, I find that a collaboration is good solution. Yes, the programmers know the specifics of what each task will entail, but in the case where there are similarities to past projects will they remember the details of what happened before and what caused things to take longer or shorter (ha! shorter!)?
Not likely, so this is where the project manager can help dig up the details and make sure all of the information is on the table before the estimate is made. I like to come up with my own guesses for some tasks and run them by the person who will be doing them, give them an opportunity to revise and then buy in that way.
And yes, MS Project will not show you the estimation history of a task, but there is a tool out there that will show you this valuable “change over time” data that you mention. I can’t help but put in a plug for LiquidPlanner’s reporting tools, and wanted to include a link to a blog post by CEO Charles Seybold where he explains how this reporting benefitted his team: http://www.liquidplanner.com/blog/2010/3/16/data-is-and-always-will-be-the-best-teacher.html
And just to add a comment to the discussion about the politics of estimating… this has a HUGE impact on project budgets. Especially these days where everyone is trimming their budgets, most times there is no point in showing a realistic estimate to the client, if you want to get the job. Lately our clients have been telling us that they want us to work on their project, but need it to be within budget X. If we can’t do it within budget X, then forget it. And, we don’t always have a lot of leeway with scope. These days its a matter of doing whatever is necessary to get the new work. So, I like to track ‘promised’ and actual, and make sure all are clear that we will very likely not be able to hit the promised budget.
Dina,
“I like to come up with my own guesses for some tasks” – yes, I would often first work out what I thought the estimates would be (using historical data) and then get the estimates from the experts. This gave me a better basis for reality checking the estimates.
“most times there is no point in showing a realistic estimate to the client” – this is a hard one. Maybe I’ve been lucky, but in every organization where we moved from “promises we can’t keep” to realistic estimates, the customers have never left us. In fact, they’ve always – always – noted that we were finally giving them good estimates. In one organization where I had finally gotten us to on time delivery, our company once again started to make unrealistic promises – in large part because our largest customer decided to “challenge” us by asking for more. This went on for awhile and so I finally decided to find employment elsewhere. At the final meeting I sat in on with this customer, their top bullet on their slides said something like “You are too nice – you don’t push back when we ask for something you can’t do.” When we never delivered on time the customer never mentioned us needing to push back on their demands. When they experienced us delivering when promised and then seeing it go away as we tried to meet their new demands – they decided they would rather have us meet our promises then “go for it” based upon their demands. This is always scary, to provide realistic estimates and information, but it has always been a successful strategy for us.
Thanks for the info on LiquidPlanner showing how estimates change over time.
Regards,
Bruce
Excellent ideas! It is very inspiring article. Thank you for sharing. Keep it up.
The couases of estimating errors are also political. We are under extrem pressure during bidding or when the business case for a project is being build to underestimate the cost so the project will get funded. once the project has started the true costs are out.
Paul,
Absolutely. My experience is this kind of pressure is one of the top 2 reasons why “estimates” are inaccurate. While I’ve often been told “we’ll lose our customers if we tell them it will take that long” I’ve, maybe luckily, never seen it happen. More often I’ve had customers tell me “it is about time you guy started giving us realistic estimates.”
Good point.
Bruce
Great article. It sounds like there needs to be more frequent adapting and estimating throughout the process so that more accurate estimates can be achieved. Remembering the past is important, but using the past to help predict the future (if your projects have the ability to do that) can be a great way to create consistency.
Peter,
I would say that there needs to be more recordings (automatically, within project tools already being used) to capture what actually happens during a project. The “recordings” I’ve used successfully include meeting minutes, archived status slides, defect tracking records, old project plans, etc. These I’ve found to be great sources for getting accurate estimates – almost always more accurate then just the memories of the participants. The best estimators usually show up with records of past efforts showing how long things took.
I’ve found most organizations to be very consistent in their performance, but it is rarely obvious to the normal observer until we’ve found a way to capture the actual detail (again, using “recordings” such as those mentioned above). During the typical hectic activity of a project, it is often hard to see the patterns, how things repeat, and how long tasks take. In one large organization it was not untypical for the experts to say “it takes us about 3 days to complete these type of tasks” only to find that the real, measured, average number of days to complete these type of tasks was 10 days.
Good comment.
Bruce
Good article.
I concur that projects could fail because of poor estimating and the unrealistic expectations to meet these inaccurate estimates. Remembering the past and keeping it all organized will definitely help producing more realistic estimates.
Predicting the future is hard and requires good facts and specifics about what to be done and how to be implemented; the more you know about how you intend to go about it then the more accurate the estimate is. Often estimates are required and ordered at very early stages with little information on what to be achieved; the past could help providing a quick order of magnitude, not estimate, to help a sponsor decides whether they have the budget for it and is worth pursuing further.
A good representation and participation of the actual project team (developers, business analysts, designers, architects, QA analysts, PM, etc.) is essential to producing a meaningful estimate. Last, it is important to review and re-estimate the project on an on-going basis.
Pierre,
“the past could help providing a quick order of magnitude, not estimate”
In many cases, even when a project has “a good representation and participation of the actual project team” they produce a bad estimate. This is common in organizations who regularly struggle to deliver projects as planned. When this conventional (and good) approach does not work as expected, I’ve found that using the past performance (usually, just the average) works very well. In every case where we’ve used it, it has out performed the normal bottom-up detail driven team provided estimate. “Out performed” means in these cases that they finally delivered a project/product on time and with good quality – where in the past they had rarely, if ever, had done so.
So past performance can provide a very good estimate – but I primarily use it in organizations that are struggling with their estimating process. Even when an organization has a pretty good (i.e., accurate) estimating process, I still suggest they know their past performance and regularly compare it to their current estimates. If one day they see a big disconnect, like a project with a schedule never attained by any project in the past, it serves as a trigger to ask the question “why is this project different.”
Excellent point.
Bruce
Timely post.
I’m going into an estimating process with a client and I know they don’t have a lot of experience estimating.
I’ve asked them to think of two estimates: if you were 100% on the project and if you were doing your day job but no other projects.
That will help me get a handle on the prioritization of other projects, but your information will help them get to the estimate.
Thanks
Perry,
Yes, it can be very enlightening to ask them how well similar efforts have gone in the past (they may be reluctant to share this – or may only provide the “last” plan, which had 3-5 replans before it). Especially for a client who does projects only occasionally (say, every few years) this past record is often a better “ball park” then any estimate they will produce. While the projects themselves may be fairly different, they are often using the same people+processes+tools that they’ve used in the past. It is a bit unintuitive, but often this people+processes+tools has more to do with how long things take (and how much they cost, etc.) then the details of the project.
Good luck.
Bruce
Great article, agree 100% with 2 and 3, minor disagreement with #1. Estimating should be made by the developer and then questioned by the Manager. Unless the developers estimate their tasks, they will never buy into the development schedules.
Charlie,
“Unless the developers estimate their tasks, they will never buy into the development schedules.”
I’ve only found this a problem when schedules are generally not very good. Using past performance in this way, the programmer’s know that the schedule is based upon how they’ve done in the past. In every case we’ve used this, the programmers would tell us something like “OK, that’s finally a reasonable schedule.” (Scrum velocity is also a good example of this – using past results to anticipate the most probable future.)
Where I’ve let the programming team set their own schedule, I’ve often challenged them to “show” me evidence that what they propose is a doable schedule. When they ask me how they could “show” such a thing, I ask them to show me several similar completed projects where they’ve done it in about the same time frame. They often come back with an updated schedule, usually a bit longer than they originally proposed, and with an attitude of increased confidence in their proposed schedule. This way we get the benefit of both past performance and buy in.
Excellent point. Buy in – and confidence in – whatever method is used is always critical.
Bruce