Absolutely. The bell shape comes from more hours and “peeling the onion” of functionality that is tested. The initial rise was when system test started. The peak was usually when field testing (testing in the customer environment – see the comment on the chart) got started. So the bell shape was indicative of this organization and how it tested. For other organizations it might be more skewed, where we see an early peak and then a decline with time as issues are uncovered and fixed.

I’ve not observed that errors reported are proportional to hours tested. My experience has been a fairly consistent rise, peak, fall – but the shape of the curve is a team+process+product characteristic. I could believe that if all defects were equally identifiable (i.e., if tested we would find and report the error) then we would see a proportional relationship. What I’ve observed is that initial defects mask other defects and so until one it fixed the others cannot be detected (and often, not all functionality is completed and delivered by the time testing starts). The 2nd major dynamic I’ve observed is that as defects are fixed, these fixes can cause other defects and so we see “fixes” generating defects (e.g., a lot of fixes is associated with a “bump” in the number of “new” defects found). The final major dynamic I’ve observed is when the software is fairly defect free (from obvious defects) then we’ll start to see more defects and fixes for long term issues such as memory leaks, complex feature interactions, network/latency related issues, other time driven issues and performance issues. These dynamics would appear to conspire to cause the rise, peak, fall, where especially the fall can be slow and long.

Great comment. Thanks.

Bruce