CLIA Final Rule

The official guide to developing IQCPs is finally out! What does it tell us about laboratory implementation of Risk Management for Quality Control? Do we even need the EP[tm] 23A  guideline anymore?

A Step-Skip-Step Guide: The New CMS/CDC Training Guide for IQCP

James O. Westgard, Ph.D.
JUNE 2015

At long last, after passage of almost a year and a half of the two-year educational period, CMS and CDC have issued a "step-by-step" guide for "Developing an IQCP" [1]. The Guide is available as a PDF and can be found by searching for "Developing an IQCP." The document is 62 pages long, full-color, with losts of lists, bullet points, and pleasing graphics. The guide describes an example application, and includes worksheets that can be used by the laboratory to create the 3 components required for an IQCP: (1) Risk Assessment Worksheet; (2) Quality Control Plan Worksheet; (3) Quality Assessment Worksheet.

The approach seems simple and straightforward, in part because CMS/CDC guide does NOT follow the CLSI EP23[TM]A consensus guidance for use of risk management for laboratory Quality Control [2]. That is hard to believe, given that CMS specifically cited EP23A for guidance in its initial announcement and description of IQCP [3].

Section §493.1250 allows HHS to approve a procedure that provides equivalent quality testing to the analytic systems requirements in §493.1251- §493.1283. In light of advancements in technology, CMS has adopted a risk based approach termed Individualized Quality Control Plan (IQCP). The Clinical and Laboratory Standards Institute (CLSI) document EP23A – "Laboratory Quality Control Based on Risk Management; Approved Guidelines" provides helpful guidance to laboratories on the development of quality control plans for test systems. Portions of the EP23A document capture the principles of our intended policies. So, as EP23A is a copyrighted work of CLSI, we obtained CLSI's permission to utilized those portions of the CLSI EP23A Guideline that captured our intended policies for IQCP under the equivalent quality testing provisions of the CLIA program.

In the end, however, CMS and CDC seem to have decided that the risk assessment must be further simplified for medical laboratories, particularly those areas that currently employ EQC procedures with reduced frequency of controls, only weekly or monthly analysis of external controls.

CMS/CDC Words vs EP23A Definitions

According to the CMS/CDC IQCP Guide, risk assessment is the identification and evaluation of potential failures and sources of errors in a testing process.

However, that is not risk assessment according to EP23[tm]A[4]. Let's look at this statement in the context of the definitions given in EP23[tm]A:

• Risk – combination of the probability of occurrence of harm and the severity of that harm.
• Risk assessment – overall process comprising a risk analysis and a risk evaluation.
• Risk analysis – systematic use of available information to identify hazards and to estimate the risk.
• Risk estimation – process used to assign values to the probability of occurrence of harm and the severity of that harm.
• Risk evaluation – process of comparing the estimated risk against given risk criteria to determine the acceptability of risks.

Risk, by definition, relates to the probability or frequency of failures that cause harm, as well as the severity of that harm. Failures occur because of hazards or failure modes that are present in the testing process. Identification of hazards is the first step of risk analysis. Estimation of risk should then follow, which means assigning values to the probability of occurrence of harm and the severity of harm. Once those values have been assigned, risk can be evaluated by comparison of the estimated risk with criteria for acceptable risk. Unacceptable risks should then be mitigated by prevention (to reduce occurrence), detection (by implementing controls), and providing physicians with information for safe use (to reduce severity).

In the CMS/CDC guide for developing an IQCP, there is no assessment of probability of occurrence or severity of harm, there is no assignment of values for risk factors, no evaluation of the expected risk, and no comparison with criteria for acceptable risk. In other words, there is actually no risk assessment.

Hazard identification, not risk assessment

The CMS/CDC risk model is described by the questions in their risk assessment worksheet:

1. What are our possible sources of errors? What can go wrong?
2. Can our identified sources of error be reduced? (Yes, no, not applicable)
3. How can we reduce the identified sources of error?

These questions must be considered for the total testing process (pre-analytic, analytic, and post-analytic phases) and, specifically, possible sources of errors in the specimen, test system, reagent, environment, and testing personnel.

There is a clear focus on identification of possible sources of errors, which is the starting point for risk assessment, but what follows is simply a question of whether you can do anything to reduce those errors, and if yes, what can you actually do. That is fine and good nad useful, but that is not risk assessment! That is simply hazard identification followed by a choice of whether or not to do something about the hazards, regardless of their potential impact on the patient.

The CMS/CDC approach is a sleight of hand solution to the complexities of risk assessment. Just call it risk assessment, but skip the actual risk assessment step and instead  do something simpler. Keep it simple, simplistic, reductive, but keep the name risk assessment to justify the approach. And recommend something far less demanding be done in laboratories. And hope that this approach will provide equivalent quality testing. But by lowering the standard of what's required, we've already doomed laboratories to develop sub-par quality, not equivalent quality.

Changes of QC Practices, NOT Necessarily Improvements

"Equivalent quality testing" is the root cause of the problem about what constitutes an appropriate QC procedure. Equivalent quality testing was the justification for EQC and is now the justification for IQCP! Actually, equivalent quality testing is another regulatory term that has no real meaning! Another name that sounds good, but has no working or operational definition. Equivalent quality? Equivalent to what?

As laboratory analysts, we have every right to be confused about what's going on with QC! The regulations have become purposely confusing, using names and terms that imply one thing, but actually represent something else. It's called "framing" of ideas, e.g., the USA Patriot Act that imposed widespread collection of virtually all communications in the US, which has now been replaced with the new Freedom Act, which mainly modifies where that information will be stored. It saddens us to see similar "framing" in our world of laboratory testing and quality control. It should cause us to pay more attention to what's happening, rather than assuming whatever is happening is for the good.

Another quick example from the CMS/CDC Guide to illustrate that new recommendations are not necessarily improvements. In the "Happy Day Physicians Group Quality Control Plan", page 38 in the IQCP Guide, the example QC Plan shows that external QC materials are to be analyzed with a frequency of every 30 days and that the criteria for acceptability are the "acceptable range printed in the manufacturer's package insert." Notice how vague that is. And recall that the use of manufacturer ranges are known to be too wide because they include significant between-laboratory variation, rather than the observed variation within a single laboratory. Using manufacturer ranges is generally considered to be a bad QC practice. See the guidance in CLSI C24A3 about control limits:

8.6 Setting Control Limits. Control limits should be calculated from the mean and standard deviation that describe the variation in QC results expected when a control material is analyzed by the measurement procedure actually in use in a given laboratory.

8.6.1 Values for the Mean and Standard Deviation. The mean and standard deviation of results for a particular control material should be established on the basis of repeated measurement on those materials by the measurement procedure in use in the laboratory. Control limits can then be calculated from the means and standard deviations observed in the laboratory when the measurement procedure is operating in a stable condition.

Of course, if you're only running controls every 30 days, it is difficult to collect enough information about performance in your own laboratory. So, in the example application, CMS and CDC recommended the use of a manufacturer's assay ranges for QC, rather than having the laboratory establish its own mean and SD for calculating control limits. If that becomes general practice, the detection capabilities of Statistical QC procedures will be degraded. Nevertheless, on page 40 of the CMS/CDC IQCP Guide, it reminds laboratories that QC must provide for immediate detection of errors for each phase of the testing process (i.e., before, during, and after testing). Does finding an out-of-control results after 30 days provide you with immediate error detection? Immediate detection of errors is not going to happen if you follow the IQCP Guide!

What's the point?

Anything goes for IQCP if you use the worksheets provided in the CMS/CDC IQCP Guide. The worksheets provide a way to document your development of a QC Plan. That may be enough satisfy inspectors, particularly if you include references to CMS and CDC as the source of the worksheets. If you follow the government instructions, how can you be wrong?

Admittedly, and despite all the flaws in the IQCP step-skip-step guideline, there is a thin silver lining to this cloud. The approach is actually a (albeit very small) step in the right direction: focus on the identification of hazards. Unfortunately, the rest of the guide is entirely qualitative and subjective, even less objective than the EP23[tm]A guideline because there is no attempt to estimate and evaluate risk. While there are ways to simplify the risk assessment methodology, eliminating the concept of risk and ignoring the factors that contribute to risk (occurrence, severity, detection) is misleading and sets back any efforts and education to understand risk and how it should be managed.

References

1. CDC, CMS, US Department of Health and Human Services. IQCP – Individualized Quality Control Plan: Developing an IQCP – A Step-by-Step Guide.

1. CLSI EP23A. Laboratory Quality Control Based on Risk Management. Clinical and Laboratory Standards Organization, Wayne, PA, 2011.
2. CMS Memo of August 16, 2013: Individualized Quality Control Plan (IQCP): A New Quality Control (QC) Option. www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/Downloads/IQCPbenefits.pdf
3. CLSI C24A3. Statistical Quality Control for Quantitative Measurement Procedures. Clinical and Laboratory Standards Institute, Wayne, PA, 2006.