Project management tools is often about turning an out of control situation into a rational, methodical discipline.  Sometimes, however, losing control of the situation can be the best course of action.

We had a major crisis.  Our software had a tendency to corrupt the database of our customers.  The development team was working to figure out why our software corrupted the database and to get it fixed.  The customer support team would correct these database problems on the customer site as they happened.

The problem was, as the director who owned the customer support team reported, we created more problems than we solved.  So the customer support team was moved over to software development, my organization.  I, the new owner of this problem, spent a lot of time explaining to everyone how this problem was a long time in the making. Don’t expect it to be fixed overnight.  Six months to a year may be needed.

In parallel with but unrelated to this event, I had sponsored a multi-department meeting to help motivate ideas to improve our software quality and process.  The first meeting did not seem to generate many good ideas, but one notion came out loud and clear.  One of the things we did well, which few people realized, is that when we fixed a software defect, it stayed fixed.   Many organizations I had been part of in the past had the problem that fixing a defect would often cause multiple other defects.  Our track record was so good that our customers had no problem with us directly patching their operational systems – without them first testing the change.

Fast forward to several months after I had picked (convinced!) one of my managers to own the customer support team along with the defect repair team that he already owned.  The person chosen to lead this team was so important to senior management that the COO of the company decided to interview and select the person for the job.  I admit this annoyed me. Maybe the COO had added value, but she only interviewed the one individual I had identified. This guy also negotiated a salary with her that exceeded my own at the time!  He was a great guy however and did an amazing job.

As I said, fast forward to several months into owning this new problem.  Problems had been continuing. We let go some of the customer support folks and were working with the remaining to help them better understand what they were doing.  After a certain problem arose, the COO said to go find the person who made the “bad” changes and make an example of them.  I was suddenly creatively incompetent and never seemed to be able to locate the exact person who “caused” the problem.

OK, really this time, fast forward to several months into trying to solve this problem.  I came up behind my defect manager and one of the customer account representatives.  They were talking excitedly about something that apparently had gone quite well.   They did not know at first that I was behind them, and I listened to what they were saying.  I was horrified.  Apparently, they had bundled up into a special release many of the defect fixes that were in the next software release which was many months from delivery to our customers.  In addition, they had also put into this release other defect fixes that had not yet even been scheduled for a release, let alone tested by our release process.  They had then taken this special release and had directly patched the customer’s operational system!  Sure, we had a policy that some well behaved fixes could be directly put into a customer’s system with the customer’s agreement.  Yes, the manager had full authority to make such patches.  No, this release did not meet the spirit of our policy or intent into what we could do in this manner with a customer’s mission critical system!  However, this patch solved the myriad problems we had been trying to fix for many months.

I will forever be terrified and elated by what had happened.  First, it was probably my doing at trying to find new ways and approaches that surfaced and put this idea into these managers.  I had put this guy in charge of fixing this problem (the COO thinks she did of course) and charged him with finding a good short and long range solution to the problem.  I realize if they had come to me with the idea, I don’t know if I would have approved it.  I would certainly have bounced it off the other directors as they would all be affected if it succeeded or especially if it failed and caused more problems.  We also had been successfully sued in the past by a customer for irreparably damaging their business with a flawed software release – luckily before my time.  I suspect there would have been a more than even chance I would have said we’ll let the full software release process complete to ensure we had the best quality product possible. This would have probably added months to finding a resolution to the customer problems we were having.

Reflecting on this experience and others like it, I realized that “my” successes were often because I had empowered folks to take the initiative.  I observed that when my teams were just slightly out of my control is when they often performed the best.

How under control is your project team?