Basic QC Practices

A recent publication of large academic medical centers in the USA revealed that 90% of them still set their control limits at 2 standard deviations from the mean, and only 10% use a variation of the "Westgard Rules". Have we really fallen so far?

QC in 2018: A Stark Reminder about the Truth Standard for Laboratory Testing

James O. Westgard, Sten Westgard
August 2018

Please note: this article contains politics

With publication of a survey on “Quality control practices for chemistry and immunochemistry in a cohort of 20 large academic medical centers” in the August issue of the American Journal of Clinical Pathology [1], the issue of the quality of quality control has again surfaced:

"There appears to be no systematic approach to defining QC rules or Frequency. The Westgard rules offer a systematic and thoroughly vetted approach for QC to detect errors while minimizing false-positive rates, and additional methods involves rules for QC—level repetition have also been mathematically studied. Interestingly, Westgard rules were used by a small minority of academic center laboratories. Most laboratories prefer home-validated QC rules, which may rely on the director’s experience and expertise rather than a rigorous validated statistical approach to designing QC rules…"

To us at Westgard QC, that is a disappointing finding, but not entirely unexpected. In 2004, we published a book “Nothing but the truth about Quality” [2] which reflected on the impact of business practices and ethics (or lack thereof) on emerging problems in healthcare and medical laboratories. At that time, it was clear that the effect of CLIA was to reduce quality to minimum requirements, or “quality compliance” as we called it. We suggested then that the maximum “Tell the Truth, the Whole Truth, and Nothing but the Truth” could be applied to laboratory testing and should also become the standard for quality. The Truth Standard can be described as follows:

• The Truth requires that a test be related to the disease process of interest and that the interpretative guidelines be understood in terms of the quality (or limits of variation) required for the test.
• For the Whole Truth to be known, the test must be measured by a reliable method having the proper specifications for precision and accuracy.
• To provide Nothing but the Truth, a test result should not be confounded by unknown or undisclosed factors, such as changes in the subject due to biological variation or changes in the method due to lack of stability and quality control.

Today, truth is on even shakier ground. We should all be concerned about how a lack of truth impacts business practices and ethics, and likewise the insidious effects that creep into all aspects of our work and our lives. We are being told not to trust anything that we see or read [President Donald Trump, Kansas City, July 18 2018: “What you’re seeing and what you’re reading is not what’s happening”]. Do you think that encourages our patients to trust their test results? We would like to think these are two different realities, but it has also become clear that it may be difficult for people to distinguish between them.

The academic medical center survey on SQC practices provides stark evidence of our need to stand up for truth and quality in the form of evidence-based best practices for SQC. Evidence in this case should include defined requirements (analytical performance specifications) for the quality of a test, documentation of the precision and bias of testing processes, and design and implementation of SQC strategies that account for the known rejection characteristics of control rules, as well as optimization for run length to minimize patient risk.

An interesting recommendation from the report is a call for laboratory professionals to improve SQC practices in US laboratories:

"The survey suggests an opportunity for laboratory professional organizations to convene a consensus panel to determine a best practice approach (or approaches) to QC in the chemistry laboratory. This would help to ensure quality testing by enhancing error detection and reduce the costs associated with excessive QC and the use of QC rules that create high (false-positive) run rejection rates."

This is an interesting recommendation, since CLSI recently published a 4th edition of its C24 guideline for “Statistical Quality Control for Quantitative Measurement Procedures.” [2] That guideline was developed by the CLSI consensus process, but evidently that process is not achieving the goal of improving SQC practices in US laboratories. There may be a variety of explanations for this shortcoming:

• Many laboratories do not have access to the guideline because of its cost;
• The guideline is too difficult to understand because of the statistical and theoretical nature of the subject;
• The guideline does not provide a practical methodology to implement the recommended principles and approach;
• CLSI has not made sufficient educational efforts to promote the guideline and recommended methodology;
• Laboratories aren’t interested in a quantitative SQC planning methodology because CLIA only requires analysis of 2 levels of controls per day, with whatever rejection criteria the laboratory chooses to use.
• Other accreditation organizations must be “deemed” by CMS, which leads to a marketplace for inspection services that is driven by the lowest requirements consistent with CLIA.

Regardless of which or many reasons have caused the current predicament, there is now the need for a professional organization to step up and fill the void, otherwise US laboratories will fall behind the global trends for improving the quality of laboratory testing processes.

Frankly, we at Westgard QC has also been baffled by the 4th edition of the C24 document and raised some of the issues in discussions in the literature [3,4,5]. To be blunt: no one can implement the “roadmap” from C24-Ed4 without additional help and resources. It seems clear the guideline was written with an assumption that certain commercial software had to be purchased. In an effort to overcome the C24 limitations, we are making available new graphical tools and worksheets that are based on our recommendations in the literature. These "Westgard Sigma Rules" provide you with evidence-based, rational, scientific, and simple recommendations for running QC.

References

1. Rosenbaum MW, Flood JG, Melanson SE, et al. Quality control practices for chemistry and immunochemistry in a cohort of 21 large academic medical centers. Am J Clin Pathol 2018;150:96-104.
2. Westgard JO. Nothing but the Truth about Quality. Madison WI:Westgard QC, 2004.
3. CLSI C24-Ed4. Statistical Quality Control for Quantitative Measurement Procedures; Principles and Definitions. CLSI, 950 West Valley Road, Suite 2500, Wayne PA, 2016.
4. Bayat H, Westgard SA, Westgard JO. Planning risk-based SQC strategies: Practical tools to support the new CLSI C24Ed4 guidance. J Appl Lab Med 2017;2:211-221.
5. Westgard JO, Bayat H, Westgard SA. Planning risk-based SQC schedules for bracketed operation of continuous production analyzers. Clin Chem 2018;64:289-296.
6. Westgard SA, Bayat H, Westgard JO. Selecting a risk-based SQC procedure for HbA1c and a Total QC Plan. J Diabetes Tech Science 2018;12:780-785.