Do We Manage Projects By Intuition or by Methodology?
I talk a lot about using project management tools and methodologies. What happens when we are not using a tool or method? What do we call it then? I call it management by intuition. Others call it management by experience. Dictionary.com defines intuition as “direct perception of truth, fact, etc., independent of any reasoning process.”
Let me admit that I love a good system or process. While I’m a creative person and like to just leap in and try things out, most times when it comes to doing something important that needs doing, I’d rather have a well defined, well thought-out, process that is followed with discipline.
I have observed that many methodologies are originally chosen when management by intuition did not serve us well. For fairly non-complex activities, just doing what seems right and natural will work well enough (home, social, volunteer, community projects, etc.). However, I’ve never experienced a project in the military, federal or corporate world where “just do what seems right” has worked well. This is where, often after poor results, we finally decide to try a disciplined process.
Many of you reading this might well be saying “yeah, yeah, this is all obvious, what is the point?” The point is that many of our projects are probably run by intuition rather than by methodology or process. Even as the project appears to be following a methodical approach, it really is being driven by the “seat of the pants.”
Huh? We either have a methodology or process or we don’t, right? Not necessarily. Think about it. How many times have we seen a team working on a project and when we ask them about some aspect of the project we get indistinct answers and hand waving? “Can I see your plan?” “Oh, it is not finished yet.” “Do you have an overall schedule estimate?” “Sure, here is our PowerPoint slide.” “What did you base the schedule on?” “Oh, we think everyone will like it if we say it will take this long.” How do you know it will take this long?” “Oh, but everyone says this is the right amount of time to take.” “OK, so what are you using the plan for?” “We need the plan because it has to be checked off on the QA checklist …” The process being followed is indistinct and being done intuitively.
Ok, but — you know — if it works who cares if a process or methodology isn’t strictly followed? “Don’t worry, we’ve done this before, everything will be fine.”
All these phrases came from a range of projects, all that ended up cancelled or late with low quality. Just about all of them commented at the end of the project that “This is how it works. Things are not perfect. Thinking that it would be on time and with few issues is just unrealistic. Real projects just don’t happen that way.”
In one particular case, with a Director of Development who was managing a product that was late and buggy (and saying many of the things above), we outlined how we had just delivered the premier product of the company on time and with quality touted by the customer. He at first didn’t realize we were talking about a real project, just completed. He showed visible disdain and scoffed that we were naive and inexperienced in the real world by talking about any such thing. He quickly went quiet when he realized we were talking about a large project — much larger than his — that he was well aware of, but didn’t know it’s whole story. Even with his long history and notable reputation in the industry with startups he had never led nor been apart of a successful large project and was suddenly speechless when face to face with real performance. The details behind his intuitively managed project are worth examining.
This particular manager had been told to go off and create the next great product using whatever methodology and approach he wanted to use. This was going to be a special project that developed not only a great product but also a great project methodology and process that the rest of the company would adopt in future projects.
I was visiting their site as part of a team that was going to adopt what they were doing into a couple of our new projects. In order to get an idea of the size of the schedule, I would regularly ask them how long some phase of the process took (e.g., getting commitments from a strategic contractor on delivering their part of the project). I would get an answer such as “Oh, it will take you about 2 weeks.” I would then ask “How, long did it take you?” Often I would then see visible annoyance or discomfort and they would finally say something like “it took us twice as long, but you will be able to do it in half that time!”
In just about every case where I asked “how long did it take you to …” I got this same answer “it should take you X” followed by a reluctantly given “OK, it took us longer, but ….” During the actual projects, it turned out that in every case the amount of time to accomplish what they had just accomplished was comparable. It didn’t “suddenly” only take half the time. It took just about the same amount of time it had taken in the recent past.
We had two key products that were going to be on this new great approach, and we finally cancelled them both as all the evidence that came in at the beginning of the project (i.e., ability to hit milestones and cost estimates) significantly exceeded the plan we had devised based upon the “intuition” of the experts who had just completed a similar project. The other evidence to cancel these two products was the fact that several other products this new methodology team was also managing, were not performing any better than the first.
These experiences helped me to characterize how typical “intuitive” project management plays itself out (which shares similarities with inexperienced project management and underestimated projects):
- Phase 1: The project starts out with great press, good looking plans, and appears to be on the fast track. Everything is reportedly going well.
- Phase 2: Milestones start to be adjusted as some unanticipated things have gone not quite right, but we should have no problem dealing with the change as everything else should be fine.
- Phase 3: Just about every milestone is off track, we’ve come too far to cancel the project. We just need to replan and then everything will fall into place.
- Phase 4: The project just never seems to complete, has been replanned multiple times, and there is always a few more things that still need to be fixed, completed, re-worked, and we just need another week. Then it will be done. Months go by.
- Phase 5: The project completes or the product goes out the door with little fanfare. Everyone wants to go work on something else. Lots of slides are generated on lessons learned and the heroic things we did to get this “really hard” project completed. Often these slides will show the anticipated project “benefits” we’ll see, someday — in the future — after one more release or update to the what we’ve already done.
Too often we manage by intuition rather than by methodology. Intuition just seems easier and is not complicated by doing hard things. Intuition is always essential in managing a project. But, our underlying approach should be based upon a methodology we understand and anchored to the brutal honesty of our reality if we want to be consistently successful.
Does your organization seem to manage projects more by intuition or by methodology?
8 thoughts on “Do We Manage Projects By Intuition or by Methodology?”
Comments are closed.
In a true reality, “consistently successful” is not possible and, in some cases, undesirable. The manager and project sampled in this article may well have been set on a no-win course.
The manager’s strengths may have been eclipsed and weaknesses magnified in a structured environment, as is the case with many successful entrepreneurs.
Projects aimed to develop innovative products, often times, succeed after repeated failures, as there’s no set path to invention.
Lastly, targets, time line, measurability are concepts that do not exist in the world of true innovation.
Shawna,
I’ve been told this often, that we can’t consistently deliver software intensive systems on time with good quality. We would then go off and do it, turning an organization around from late and buggy to on time with good quality. While this site is dedicated to giving examples of how and why we could do this, one of the fundamental insights to accomplishing this is first just getting the schedule right (http://pmtoolsthatwork.com/get-schedule-right/).
Organizations that routinely deliver late and buggy do indeed have managers that “have been set on a no-win course” and breaking this pattern is key to helping an organization. Bad management habits are generally the reason one is set on a no-win course and these are avoidable. The trick is changing the habit and mindset, mostly by finally just having a successful project (http://pmtoolsthatwork.com/software-development-repeatable-process/).
“True innovation” from first principles is probably a small single digit percentage of what any organization does in its institutional lifetime. The rest are incremental innovations and in these environments, at least in the ones I’ve worked in and with, getting to on time delivery with good quality even with our latest and greatest new innovative product could be done predictably, once we got past the entrenched belief that it could not be done (http://pmtoolsthatwork.com/why-our-innovations-fail-us/).
Good comment. I don’t know how I overlooked it for almost 5 years.!
Sagar,
Yes, I often talk about “balance” in everything we do. Sometimes when we advocate something folks enthusiastically go too far and we have to help them find a balance.
Thanks for the kind words.
Bruce, thanks for the thought provoking article and the consolidation of comments from around the web.
My personal belief is – Intuition or years of experience whatever you call – Yes is used a lot. One cannot be successful only with methodology and methodology doesn’t provide decision to be taken. But intuition does. There are many methodologies and many different scenarios we face in projects, methodologies guide, but intuition helps to decide which is the correct methodology or what tweaking to the methodology is correct.
Combination of both is very important. Only intuition cannot guarantee success and same goes for methodology. The ability of the project manager to make the right combination is very important.
More comments from around the web:
Itamar Domb • I would suggest that intuition and methodology should depend on the size of a project. Projects size of up to 5 people can be managed by intuition only.
Projects size of more than 30 people can be managed only by methodology where intuition helps here and there, but it should be based on methodology.
Projects size in between will be managed by a mix of methodology and intuition. The percentage of intuition vs methodology depends on the size of the project.
The numbers that I qutoed (5 & 30) are not set in stone. It depends on the experience of the project manager and how good he/she is.
Follow Luiz Cezar
Luiz Cezar Giacomazi • Intuition, years of experience, this is what really matter, tale a teacher with no experience and put him/her as project manager and will see the results. Methology helps, increase productivity and control but in order to face day by day problems just past experiences would really help. Nice chat.
Bruce Benson • Itamar,
I like your notion. The bigger the project the more methodology is needed to keep it running well. I do tell my small project managers to use as many techniques as they can so they can get the experience with the techniques, but not to allow them to overwhelm the project.
As to the number of people, I’ve had some pretty critical software efforts that had a small number of people but we had to use as much methodology as we could muster (requirements traceability matrix, etc.) to keep track of the critical details. But I buy the general notion that technique is often needed to manage projects with large and/or changing amount of detail.
I also like to have as much methodologies and tools being used as reasonable, just in case I lose my very good project managers (often because they were very good and found new opportunities). This allows the less experienced project manager to step in and more quickly come up to speed on the project. Managers who love to keep all the details in their heads (or hip pocket spreadsheets) are often managers who temporarily appear more capable than they really are.
Thanks for the comment.
Bruce
More comments from around the web:
Nurit Zamir • What we call “intuition” is actually in most cases your years and years of professional experience naturally guiding you
Umamaheswara
Umamaheswara Polavarapu • Methodology for better communication, monitoring and control across the organization.
Lars Enstrom • I have followed the discussion on project management for some time and what strikes me is that the discussion on management of expectations is missing.
Project execution today is the same art form as it was in the beginning of the 1970ties when I was involved in my first project. You always deal with a moving target where the definitions for the project are made based on yesterdays facts.
Objectives change as do management focus, as a project leader you have to have the ability to manage and to foresee the adaptations needed to deliver towards expectations rather than towards objectively quantifiable criteria’s of yesterday .
As Tricia and Nurit state this I call intuition which of cause shall be coupled with a good methodology.
The project manager can however never be allowed to “hide behind” the methodology in order to avoid managing expectations.
Gaurav Virdy • I agree with Nurit that our intuitions are nothingbut years and years of experience. But if you try to take your intuitions and put this on the blueprint of methodology you would certainly be able to map the rationale behind your decisions are falling inline with the methodology that is often suggested for project management. Obviously for this you would need right methodology in place. I woudl even go to the extent that if you think that you are not able to map the rationale behind your decisions to the methdology then the methodolgy needs to be reviewed.
So my answer to the question is (looks like a Miss Universe answer), the best project management is to work by intuition and then map that intuition to methodology to corroborate your decision. Yes, its time consuming but it gives you better sense understanding of how you are placing your decisions on the dart.
Sandra Agolia • Intuition works with experience, but methodology is important to make sure that you are not missing anything
Arthur Roy • Intuition is better due to human factor involved and mainly applicable for service industries. If I compare managing technical components of NASA’s launch operations with managing customer support reps of a credit card industry, managing by methodology would be more applicable to the former case.
Artak Antikyan • The Intuition is becoming to methodology after defined time.
Bruce Benson • Absolutely agree that experience and the use of methodology helps improve our “intuition.” Too often however, I’ve observed managers working “intuitively” — it just seemed the right thing to do — that created great mischief. The way I usually “ground” these folks is by bouncing their ideas off a relevant methodology.
I worked with a manager, a very bright and hardworking person who was considered the managerial star in the team. The team was doing poorly, making mistakes and they were hard to rely upon. Our software projects were always late and being “buggy” was just normal. She was seen as holding the team together and she was seen as the reason why anything ever got done successfully — given how bad the team was.
One intuitive technique she used, for example, was that if a programmer made a mistake in their coding, she gave the broken code to someone else to fix. Her stated logic was that since the first person got it wrong, it was best to give it to someone else to see if they could get it right. She also wouldn’t tell the first person there was anything wrong with what they did, because she didn’t want to discourage or demotivate them.
Now to me, these were just obviously not the right things to be doing (I was in my mid 20s at this time – what did I know?). But amazingly, the organization didn’t see it that way. She was holding everything together. Without her we would be in worse trouble.
Well, I eased her out of the center of things. Programmers became responsible for specific areas of code that now belonged to them. What happened? They learned. They got better. Their confidence improved. We went on to deliver an annual reoccurring project a month early and with no errors detected by our customer within the first six months. The team had never done that before and the notion that something like this could ever be done was considered naive by management. The system was too complex (military space-borne early warning) and our team was too inexperienced (young military with a high, 33%, annual rotation rate) too do much better than we had done in the past.
Intuition is still essential to doing the right thing at the right time, but it is useful to periodically check our thinking and approach. There is always more to learn.
Great feedback, thanks!
Bruce
Carol,
I thought about naming the phases (e.g., perfection, tweaking, scrambling, chaos, denial/distancing – kind of a reverse maturity progression) but thought it was a bit too judgmental and I just wanted to relate objectively what I had experienced.
Thanks for the feedback and I’m glad they are thought provoking.
Bruce
Bruce,
You always write insightful posts, but today’s was particularly good. Process-centric rhetoric seems to prevail these days (agile, kanban, scrum, even waterfall all name and brand their particular “processes”) – yet it is the human understanding and interpretation of said processes (i.e., intuition jumps in) that seems to prevail on projects.
We get things done by DOING things right (whatever the process is called) and working with people – not by talking about which process(es) work best. I love your five phases – maybe naming them would result in a new SDLC and a whole set of processes… (LOL).
Keep up the posts and thanks for sharing them with the world on your blog!
Carol