Project Managers Repeat After Me: Software Development Is A Repeatable Process
I had a job interview for VP of Development. The interviewing VP with a long history in the industry asked me a simple question: Did I think software development was a repeatable process? I said yes. I won’t describe the ensuing discussion except to say I didn’t get the job. We didn’t even get to talk about project management tools!
I’ve heard similar things such as Six Sigma can’t be used in software development. No two software efforts are ever the same so they can never be done the same. Every software development is a science experiment.
Balderdash.
I tell people that for over 30 years we’ve delivered software projects on time and with good quality (for details, see Is Your Project On Time?). Do you think anyone ever believes me? How about after we actually do it? Early in my career folks would tell me “you just got lucky” or “you got an easy one.” Later in my career I’d hear folks months and years after we delivered on time say things such as “we never did figure out how that was done!”
Whether it is a waterfall approach or an agile methodology it is a process and it is repeatable. We do it every day. When I hear that software is not a repeatable process it usually comes from folks who have a hard time delivering working software when needed. They confuse the inability to deliver working software when promised with the notion that it is because every software effort is different. We can’t do exactly the same thing on every software project, so we don’t know when it will be completed.
These folks are confusing the process with the details of the project. Driving cross country is a repeatable process. If I drive to Disney World or to Disneyland the approach is the same: Plan, drive, arrive. The fact that the details are different, we go in different directions to get to each location, we stop at different places along the way, we pay different prices for gasoline and food, we drive at different speeds depending upon the speed limits, are all just details of the requirements and the execution. How we go about turning the requirements into a delivery, of our family, to each location has the same overall process and we can use the same tools in the same way. The same applies to software intensive projects.
Challenging Obstacles Instead Become Deadly Hazards
What is often not seemingly repeatable is delivering a project on time. This is another case where we frequently confuse ineffective planning with problems of the process. If I want to drive to Disneyland from Chicago and give myself one day or just one tank of gas, I’m probably not going to make it. Using these kind of questionable project parameters makes it seem like the traffic lights, construction, speed limits, the size of my fuel tank, and other vehicles are all conspiring to slow me down — but these are just normal factors in such a drive. They only become critical factors when we do something silly like try to drive 100 MPH to get the job done. When driving significantly over the speed limit other cars become a greater hazard. The chance of being stopped by the highway patrol becomes a risk. Suddenly slowing down for a traffic jam around a city or major intersection or construction becomes a high risk (and they guy right behind us trying to get to Legoland in a day threatens to plow into the back of our car). The point here is that when we set project parameters (cost, schedule, scope) that are unrealistic relative to the performance of our organization then normally solvable obstacles instead become deadly hazards to project success.
Yes, of course, if we took an airline we could probably get there in a day. Heck, if we took the corporate jet and flew directly — being picked up and delivered by police escorted limousine — we could be there in a few hours. The problem here is that folks like to believe if they drive their teams real hard they will succeed which is often equivalent to believing their car can perform like a Learjet if they just drive fast enough. I’ll buy a new project management tool (e.g., Porsche) and adopt Agile (new driving technique) that will get 10x productivity (can go 200 MPH) and we’ll make it, this time. Really. I promise. Check my math.
Now of course, someone always asks, well — why can’t we go that fast? Prove to me that we can’t do it in six months. Just because we’ve never done it before nor has anyone else, why can’t WE do it? This is where folks confuse the notion of improving the process with the current capacity of the process. Why should it be so hard to just do things 20% faster? Many of the organizations I worked with would have achieved 100% on time projects if they could have just delivered 20% faster than what they had done in the past.
For more on understanding current capacity see Lessons Learned From Marathon Runners
Why didn’t they? Because it sounding reasonable does not mean it is not horribly difficult to do. Horribly, because when I suggest that we — while also trying to improve our speed — just start the new projects 20% earlier and use the average schedule … this is usually considered impossible: “We don’t know enough at that point in time! Everything will change too much if we start that early! It is too late for our current projects! We have other things we need to do, we can’t start on it now!” This is one thing that is relatively easy to do (we’ve done it) when compared to figuring out how to get a 20% increase in speed in the current project. We do want to increase productivity and speed. The silver lining, for example in just starting the project on time, is that it also empowers improvement efforts to actually have an impact.
For more on how accurate schedules encourage improvements, see In Project Management 9+3 Is Not Equal To 12
Software and many other types of projects are accomplished using a repeatable process that works well — when it is understood and managed well (see The One Perfect Project Management Methodology). Too often we confuse the problems caused by aggressive schedules, insufficient budgets, and excessive scope with problems in our process or methodology. I always recommend first finding an objective way for setting these overall project parameters (schedule, cost, scope). But, for whichever approach we take, if we get these parameters in the right ballpark then our projects and our project teams will be — repeatedly — successful.
Are your projects being managed using a repeatable process?
12 thoughts on “Project Managers Repeat After Me: Software Development Is A Repeatable Process”
Comments are closed.
Bruce – well said! Of course software development is a repeatable process – it fits exactly as a semi-closed or ‘making a movie’ type of project in Eddie Obeng’s model from his book, ‘The Project Leader’s Secret Handbook’; stakeholders/customers are unclear about the project’s objective and detailed requirements but they understand the process of software development.
As you say, it is typical that senior managers set impossible parameters (or as I would call them, constraints) and then expect the PM to ‘just do it’.
The ‘just do it’ culture is prevalent in most organisations that I have worked with. It doesn’t matter what process or method you follow, if the organisational behaviours are poor then unrealistic and infeasible constraints (deadline and budget) will be seen as the only performance management tools in the toolbox.
Projects don’t go wrong at the end, they go wrong at the beginning. What’s needed is a recognition that ‘planning’ is a valuable thing to do. Your suggestion that we should start 20% earlier is, of course, sensible; it would enable ‘proper’ planning whilst acknowledging the need for the PM to adopt an ‘agile’ approach as Frank suggested.
To stick with your analogy, the plan is the map for the journey you are about to undertake; the schedule is the chosen route using your chosen method.
Regards
David MacLeod
I agree with the premise that deterministic processes are repeatable. The non-repeatability usually boils down to implicit ill-formed requirements. System Y is a superset of System X that does B plus everything else. Well if no one truly knows what system X does who can write the requirements for system Y.
Now lets take your cute analogy of driving to Disney world except now its the 1800’s and Florida is controlled partially by the Spanish and partially by Seminole Indians. If you venture down the wrong path you get scalped, but you heard about a guy who managed to do it successfully. Is this repeatable, was he lucky?
For all your assertions about doing whatever you did you have to realize it was a team effort and your role as project planner might pale in comparison to the Indian scout who knows the lay of the land and kept you out of trouble? In short easy/trivial projects are highly repeatable as is brushing of teeth or putting on pajamas. Difficult/interesting projects have lots of moving parts and they typically carry a non-zero probability of failure. Many times the source of this failure is outside the scope of the project or at least outside the scope of project management/budget/timeline.
Notice I am not saying it can or cannot be done the issue is uncertainty. Engineers are culturally biased to avoid uncertainty. This is usually a good trait when the uncertainty can be replaced with a deterministic solution but sometimes the uncertainty is at the core of the problem and cannot be circumvented.
As for six sigma. If your little x’s and big Y’s help you sleep soundly then by all means enjoy. The point of my tirade is not that planning is not a worthwhile endeavor but rather you should plan but you must also be able to think on your feet, because the best laid plans of mice and men …
Frank,
I worked for companies that for years could not get their software out the door on time, on budget and with the kind of quality they wanted. The reasons stated were usually scope creep, customer changing their minds, wrong choice of technology, inexperienced staff, too many unknowns in the requirements, etc. With the same people + process + tools – bad habits, we then delivered on time and with good quality. The most common bad habits were usually overall schedules that simply were not realistic. How did we get a realistic schedule? By deterministically looking at all the details and working out an overall schedule? By finding an expert who “knew” the answer? Nope. We usually looked at their track record for their last N projects (which usually were all different) and calculated the average time it had taken them. It was almost always the case that their project estimates were less than what it finally took them to complete their projects. We then said we’ll use the existing average for the current project (which was always more than their current estimate). We did miss a few delivery dates by a week or two, but that was nothing compared to the months (3-4, even 6 and 9) they were normally late.
The point is that projects are generally non-deterministic in nature but there are techniques (including using simple averages, but also Monte Carlo, Putnam, etc.) that readily master the “problems” of things that are not deterministic. In fact, my experience is that the bigger the projects (such as complex software intensive consumer electronics) the easier they are to manage by using averages, trends, and sigmas then by trying to spell everything out in Microsoft Project linked tasks.
Each of these companies had experts who worked like crazy but the teams never delivered on time. Not even close. Once we got the project parameters, the schedule in particular, in the right ballpark they almost always delivered on time. The problems were not the process nor the talent. The problems were just bad habits – usually bad planning. Suddenly, software development which had not been a repeatable process (and everyone “knows” that – it is just a fact – it has always been that way!) became consistent and predictable.
I liked your the 1800s analogy. Good comment.
Bruce
100% agree! Having developed software since 1972 and since then managing projects, I still come across people who disdain planning and/or refuse to learn from the past. Classic example: we went for a 4000+km trip in the Australian outback with a friend who scoffed at my “excessive planning”. In the middle of the journey he admitted that on a previous trip he (and his mates) had to sleep in a river bed (through not planning accommodation)!
One caution: when embarking into a new area (though nothing is completely unknown) there is an element of “research” and greater uncertainty, requiring more risk management and more contingency.
John,
Yes, I also find folks that often don’t always want to do the research or know where to look. Sometimes the best thing to do is just wade in and start doing, but I’m sure I’d not want to challenge the outback that way!
We were visiting Haleakala crater in Maui Hawaii and my girlfriend at the time said we should just go down and hike across it. Hours later, passing people who were only there with backpacks, food and water, we climbed up the scree on the opposite side (she was in dire straights – I had to talk her up the climb). We made it to a road with no transportation to get us back. A lone car finally came along and she just jumps out and waves. They pull over and we get a ride back to where we started. Some folks are just more spontaneous than others – thought I often wonder if they will survive long!
Good story.
Bruce
African elephants are huge, to say the least.
However, they also have a believe system that defies all logic. As a small baby elephant, the owner will pen down one leg of the elephant so that it can not get away. As it grows up, there comes a time when the elephant can very easily just walk on, no dinky pen will keep it chained to one spot. Yet it believes the chain is unbreakable. Total conditioning.
I would venture to say there is some parallel between the VP that interviewed you and that elephant. Both believe to their inner core that it is impossible – and so it is.
Leonard,
I worked with one business manager who fought long and hard against using a schedule based upon past performance (i.e., a longer schedule than planned, but not longer than how long it has taken in the past). At the end of the project, when we delivered on time and with good quality, he remarked: “Hey Bruce, this stuff really works!”
So I don’t presume that folks have a completely fixed outlook, even if they have strongly opposed trying something new like this. At lot of them may not have seen a successful project and so they stick with what they know until they are convinced otherwise. I find I have to be careful to not put them in a position of saying something like “This will *never* work” or “It is *impossible*”. This only makes it harder for them to say “Hey, it does work!” if they’ve put too much of their reputation on the line when opposing it.
I’ve not met any elephant pinned managers yet, but some of them have came close.
Thanks for the great example.
Bruce
Yes, the process is the same. But you said “plan.” If project managers don’t plan correctly, it will not be done on time. A point that you made in your post. It is easy to say that software development, for example, can always be done using the same process. It is very difficult for a majority to achieve. However, it is not impossible, as you have proven. Do you think you have a more effective team that others might have?
Laura,
Actually, no – I don’t think our teams were any more capable than most teams in the industry as a whole. The proof of that is in all cases we took existing teams and achieved what they had not been able to achieve in the past. I often argue that what management must do is set up a team for success. That means figuring out the right ballpark for the cost, schedule and scope parameters of the project. Just get it close – and close can generally be “checked” by looking at any data one has on past performance of the teams (from any and all projects if nothing else is available).
The plan is important, but often not by making it extremely detailed – which is a very typical way to “fix” plans that have not worked well in the past (for more see: http://pmtoolsthatwork.com/project-management-or-death-by-detail/ ). Just as often, the key parameters of the project are consistent (as to how much time, how much money and how much “stuff”) but since these projects are, for example, always late – the parameters remain the same, but we are certain that if we just manage it closer and require more detail up front and more detailed tracking during execution, we’ll avoid all the problems in the past. I’ve never seen it work out this way. Never.
Because of this, projects not performing well, people will say it is simply because some project domains (such as software) are too unpredictable and hence there is no repeatable process that will bring them into control. Too much is unknown. Too much is different each time.
The humorous part is to somehow contrive to get a reasonable schedule – not change anything else – and watch the wonder as the team delivers on time, after a typically chaotic appearing process, and the customer is surprised by the quality of the product they receive (as compared to the past). Suddenly, all the “classic” excuses and explanations no longer make sense (e.g., scope creep, new technology, poor project management, etc.) because we delivered on time, with good quality, and all those alleged “causes” didn’t result in failure (though it made it tougher to do than it needed to be).
Good point.
Bruce
Perry,
Agreed. Many folks have a hard time spotting the similarities between projects (software or others) during the chaos of execution. The details are always different, but the approaches are fairly consistent – regardless of the details.
Thanks
Bruce
It’s not just software projects. One of the challenges is the definition of a project – something unique. People tend to look at that word and translate to never done before at all.
Each project is unique but there’s only a few ways to build a bridge, change a service model or develop a project. All projects have core similarities that can be used to more accurately estimate scope, time, and budget.
Great post.