If everyone who takes a test is getting an A, how hard can the test be? This is the question that occurred to me yesterday watching a fine presentation by Dr. Paul A. Hall, Ph.D. Dr. Hall was the speaker who presented immediately before I did at the Almond Board of California’s 13th Annual Food Quality and Safety Symposium. Dr. Hall was addressing a question we have asked here many times- why have we seen a string of foodborne illness outbreaks from facilities supposedly subject to third party audits? The recent Salmonella outbreak connected to PCA peanut products is a shining example.
One answer? Auditors are not giving out bad marks. Perhaps based on a desire to be hired again, auditors are handing out A’s like candy on halloween. Take a look at this chart from this study of food safety audits: (the study was conducted in Europe)
Audit Company | Score 90-100% | Score 80-90% | Score 70-80% | Refuse certification |
A | 98.6 | 1.4 | 0 | 0 |
B | 98.3 | 1.2 | 0.1 | 0.3 |
C | 95.3 | 3.9 | 0.2 | 0.6 |
D | 93.7 | 3.6 | 0 | 2.8 |
E | 92.9 | 2.9 | 0 | 4.3 |
F | 92.7 | 5.4 | 0.2 | 1.7 |
G | 90.4 | 7.4 | 1.1 | 1.2 |
H | 86.3 | 1.4 | 0 | 12.3 |
Average | 93.5 | 5.0 | 0.5 | 1.0 |
There was an auditing company giving its highest rating to more than 98% of the time. Overall, more than 93% of audits resulted in the highest rating. If, as I suspect, results like this would also be found in the U.S., it would help explain why plants that have "superior" ratings are the source of food recalls.
For third-party audits to mean something, the possibility of negative feedback, and poor scores must exist. Industry has to find a way to increae the independence and effectiveness of third-party audits. Otherwise, they are a waste of time and money.
The chart is from "Checklist Governance, Risk Oriented Audits to Improve the Quality of Certification Standards in the Food Sector" F. Albersmeier, H. Shulze, A. Spiller, G. Jahn. (2009)