There are some great project management tools and techniques such as Six Sigma and Monte Carlo that can boost your productivity. Sometimes these techniques, if used too early, may be too strong a medicine for your organization.

I often advocate fairly basic techniques for understanding how a project or process is going. Often I’ve been involved in some energetic discussions on the value of measuring in basic ways such things as project schedule durations and software defect repair rates. In many cases it seems the basic approach was not compelling to folks because … it was too simple. I’ve found this reaction to be all too common.

Software Defect Repair Question Starts It All

In a company I worked for I pioneered some deep insights into how well we performed in repairing product defects. I received an award that literally stated “For Knowing More About [Defect] Trends Than Anyone Else In [the Company].” These insights however were based upon very basic measurements.

I simply collected up all the times it took to repair defects, averaged them up by month, and charted them out month by month. Wow. It was taking us about fourteen days on average to fix a defect. To fix 95% of the defects that came in on any given day was taking us about 28 days. We were faster at defect repairs towards the end of the project then we were at the beginning. The trend showed a consistently increasing defect repair rate over the development process for every product.

Nevertheless, if I asked a development manager how long it generally took to repair the typical defect, I would get an answer of from three to five days. So clearly there was a disconnect between the daily perception of how long it took and how long it really took. So the solution was simple. Keep track of how long repairs were really taking during development and use that insight when making decisions and commitments. Simple, yes? Yes, it was too simple. We need to do something serious it was decided. Something on steroids!

Six Sigma To The Rescue

To improve our project management the company decided to revive its old interest in Six Sigma. Everyone was to be trained. Once everyone was trained, all our insights and decisions would then be based upon rock solid evidence and hard data. Numbers would now have meaning and would be accurate indications of how things were going. Improvement efforts would result in real and lasting improvements.

OK, I like Six Sigma. I had been thoroughly trained by the Air Force in Total Quality Management which included extensive work in Statistical Process Control. I loved this stuff. I was happy to earn the various colored belts in Six Sigma. I was working on the premier product for our company at the time, so I would have plenty of opportunity to employ the techniques. Since Six Sigma was a company initiative, people would understand where I’m coming from and they wouldn’t freak out over the numbers and the charts. Great. I’m looking forward to this. I enrolled in and completed the on-line Six Sigma Green Belt course.

Oops. I have to sign up for a Six Sigma project to complete my Green Belt. I need my boss’s approval? I thought this was a company initiative? OK, I get my boss’s approval to “go for my Green Belt” and I attend the Six Sigma meetings.

The Six Sigma meetings are eye opening. They don’t want me to work on my project. They want me to give them one full day out of my week to work on various projects they already have. None of these projects deal directly with the critical processes for my product.

I attend a few more meetings and listen to what is being worked on. I make a proposal for a Six Sigma project using my product. I outline the problems we’ve identified in earlier product developments. I show how we intend to measure and where we should be able to improve significant parts of the project process. I’ve done a lot of this already on other products. We’ll make it a formal Six Sigma project.

Nope. They’ve already told senior management what they were going to work on. Senior management told them they could not start any new projects until they complete and show benefits from the ones they’ve already started. A little disappointed, I unenrolled from the program citing a current lack of time to work on anything other than my own project.

Over a year later, we delivered the company’s premier product on time and with good quality. It was the first time in recent corporate memory. The Six Sigma analysis helped steer us on a successful course. The once skeptical business manager for the project had exclaimed: “Bruce, this stuff actually works!” The other new products adopted our approaches. The improvements, however, were not associated with Six Sigma. I never once mentioned Six Sigma during the entire effort.

My team and I remained Six Sigma beltless. There were many folks running around with Green and Black belts. There were master Black Belts.  The Six Sigma initiative melted away as senior management’s enthusiasm waned due to a lack of useful results from the formal Six Sigma projects.  The great idea of using Six Sigma to improve our product delivery process withered away due to an overdose of formal technique and cumbersome management.  (Also see the related article Seven Ways to Make That Project “Silver Bullet” Work.)

Monte Carlo Scheduling To The Rescue

We always missed our project delivery schedules. Missed them by three to four months. Always. I proposed and then demonstrated, by delivering products on time, that doing nothing more than picking the average time it had taken to deliver recent past products provided an aggressive but doable schedule.

But that was too simple. It did not take into account all the differences that exist between products! We needed something more sophisticated than averages. We are going to use a Monte Carlo technique for setting our schedules! (See http://en.wikipedia.org/wiki/Monte_Carlo_method)

OK, I like Monte Carlo. I first encountered it when I was working on my masters degree. I used it as part of my masters work to estimate how long it would take for us to clear up an eight year accumulated backlog of defects for the military software system I was managing. The simulations, using the distribution of actual defect repair times observed over the past few years, consistently showed it should take two years. Guess what? It took two years. Cool. This should be wicked to use on our current development.

I sit in on the Monte Carlo scheduling meetings. We go through the standard detailed schedule for a product and estimate each task, one task at a time. We provide a high, low, and most probable task duration. The room is full of experts. We argue over how long things take. No one has seemingly come to the meeting with data. No one is saying “over the last X project this has taken Y amount of time.” When no one has a clue about a particular task in the standard detailed schedule we just use whatever had been used in the past and provide a high and low number that averaged back to the standard number.

This goes on for a few meetings over a few days. Fewer people attend the meeting each day. Finally, I’m sitting silently in a room with just a couple of people. A project manager is quickly filling in the rest of the schedule values with the standard values. We are done.

We finally see the results of the Monte Carlo simulation at the weekly product status meeting. The Monte Carlo based schedule shows the product will only take two weeks longer than the standard schedule. You know, the standard schedule that always results in being three to four months late. Everyone oohs and aahs (imagine a Dilbert comic strip). We now claim to have a “high confidence” schedule. Sure, it is still a shorter schedule than anything we’ve accomplished before, but the Monte Carlo simulation “confirmed” the most probable end date. The inputs were provided by the experts. This time we’ll make it!

All the new projects using Monte Carlo showed a strikingly similar two weeks added to their original schedule. Not a single product shipped on time. The average time to ship was 3.5 months late.

I doubt if very many people managing or overseeing these products could accurately describe how the Monte Carlo method worked. However, these same people could engage in a vigorous debate on the merits of using the average duration, from just completed similar projects, to set a schedule. They could argue, agree, and then observe with satisfaction that the product was finally delivered on time and with quality.

Sometimes we have to start with basic management techniques before moving up to the more sophisticated approaches. Many improvement frameworks recognize the need for an organization to work its way up a “maturity” ladder. If our organization is finding that steroid techniques — such as Six Sigma or Monte Carlo — are not showing benefits, we may want to try using simpler averages and trends to understand our projects. Once we’ve done this successfully, then we are ready to move up to more powerful tools and techniques.