Activities executed by people, machines, and software leave trails in so-called event logs. What events (such as entering a customer order into SAP, a passenger checking in for a flight, a doctor changing a patient’s dosage, or a planning agency rejecting a building permit) have in common is that all are recorded by information systems. … Business processes thus ought to be managed, supported, and improved based on event data rather than on subjective opinions or obsolete experience. Process Mining, Communications of the ACM, August 2012.
Amen. Sorry, but things like lessons learned meetings and retrospectives or initiatives to implement metrics just don’t have the same impact as simply using the data that already exists. Using what I call “naturally occurring metrics,” data that comes from the actual work being done and often created by the tools we use, is almost always more reliable and more predictive than trying to remember what we had done in the past or asking people to start collecting or recording data.
One day the software build and integration team announced that they had gotten faster and better at doing their part of the process (e.g., software builds in this case) so now everyone else needed to speed up (e.g., development feature deliveries and defect fixes). This was to help reduce the total cycle time of getting a new software release out the door. The sore spot was that the build team got the Corporate VP to make this announcement. It seemed to make everyone else look bad.
Well, it turns out that the build team was correct. Their process had indeed gotten faster. What they didn’t know was that the development teams had also sped up and in fact had sped up more than the build team. So as a percentage of the total cycle time of a release, the build team’s portion of the whole process was taking longer than it had in the past!
How did we know this? I had been pulling data out of the build process logs and out of the development process (e.g., bug fix database). As a project manager all I really wanted was to correlate when a bug fix went into a release and, as was normal for me, to figure out the average time it was taking for this to happen. I didn’t ask anyone for a regular status report on how long things were taking, instead I just used the data from our build and development systems to capture and calculate it.
For more on simple averages see: Your Average Is Powerful
No, it was not easy to get this data. It was all there, it just had to be coaxed out (found, parsed, interpreted) and aligned with other data before it would divulge its secrets. In many ways it was a science experiment to extract the data, figure out what it means (i.e., how it reflects the real world) and then understand what it was telling us. I was mining the data that already existed.
“Process discovery often provides surprising insight that can be used to redesign processes or improve management ….” Process Mining, Communications of the ACM, August 2012.
I once spent an afternoon in a room full of archived defect reports. Yes, these were paper reports of past defects bundled into folders. These were the forms used by a military organization to keep track of defects moving through the repair and test process. I am fairly certain I was the only person in the history of this military unit to ever spend any time digging through all this data. The results? We finally understood the nature of the defects we were making (i.e., what dumb mistakes we kept repeating!). We even got to a point where our test team went into a panic because they could find very few defects, and no serious defects, in our software releases.
Most organizations probably have all the data they need to understand how their process performs and where to improve it. We just need to dig the data out and make use of it. It won’t necessarily be easy, but it is almost certainly worth the effort and will often provide more reliable and more comprehensive insight than trying to implement new metrics, new reporting or new meetings.
What information systems used by your project might contain useful project management data?