Dealing With Human Error
As Smith outlines in his written testimony, the Department of Homeland Security’s Computer Emergency Readiness Team (CERT) sent Equifax (alongside many other companies) a notice on March 8th, 2017 about the vulnerability in certain versions of Apache Struts. Equifax sent out an internal mass-email, which should have required its internal IT team to fix the vulnerability within 48 hours, but that didn’t happen. An automatic scan for vulnerabilities on March 15th also failed to indicate that Equifax was using a Struts version that had the vulnerability. … During his testimony, Smith identified the company IT employee who should have applied the patch as responsible: “The human error was that the individual who’s responsible for communicating in the organization to apply the patch, did not.” Engadget, Former Equifax CEO blames breach on one IT employee, Oct 3, 2017.
Our technical support team seemed to generate more issues than they fixed. They were so bad at times that a customer would rather go with an underperforming system then ask them for help. After one bad fix to a customer’s database, that brought the customer’s business to a standstill, the company COO moved the team to my department and told me to fix it, but first she said to go find the person who made the mistake and make an example of them.
Instead of looking for a culprit, we instead worked with the team to help them improve. Their previous management simply didn’t know how to manage a technically oriented team. We then never had another major problem in that team. We even patched one customer’s live system with a huge out of cycle update, that made their system so stable that we had very little we needed to do on a regular basis. We could now focus on value-added activities, that we could bill for, rather than constantly fixing our flaky system.
I like the Dr. W. Edwards Deming notion that “management owns the process” as a way to frame many problems. If something goes wrong it is very often a domino’s fall away from other key management activities. The easiest fix for many an organization was to simply replace management who had fallen into these bad habits (e.g., finding someone to blame instead of looking for root causes). Also, the Equifax security senior manager being a music major suggested to me someone who knew how to move up in an organization, and may have been a very good general manager, but who probably didn’t have the foundational competence and work experience to handle the hard parts of a job like that one.
Are you falling into the trap of blaming individuals for errors that are actually caused by poor organizational leadership?