Quality Standards

One of the disappointments of the 2023 AACC/ADLM was seeing Siemen's benchmark their newest Atellica benchmark against the oldest CLIA goals.

The Challenge of CLIA: should a new instrument be measured with 1992 or 2024 goals?

August 2023
Sten Westgard, MS

Since July 2022, laboratories and diagnostic manufacturers have known that CLIA was tightening their proficiency testing criteria in 2024. The size and scope of the changes in goals are not mysteries. That gives everyone plenty of time to adjust their posters submitted or presented at the July 2023 conference. To fail to update the analytical quality assessment of a method  - one year after the new regulations have been announced - is probably not a mistake,or a lapse, but a choice. That a major diagnostic manufacturer might claim ignorance  or choose to continue to claim ignorance of the new CLIA 2024 goals does not inspire confidence in their offerings.

Siemens announced their latest instrument Atellica ci 1900 with a chorus of posters at the AACC/ADLM 2023 meeting in Anaheim. But they used the oldest US goals, from 1992 CLIA, to calculate their analytical Sigma-metrics.

For some analytes, CLIA goals haven't changed - for example, Chloride and Sodium. For others, CLIA is newly regulating them, or shrinking the goal by 20-40%. Big changes that will have big impacts to pass rates of proficiency testing, and therefore significant impact on QC for those assays.

The table pools together some of the data from all three posters. The Sigma-metrics that changed significantly are highlighted here.

Put a different way, this is what Atellica says their performance is (when they use 1992 goals):

Looks pretty fantastic. (Please note the GGT goal is from the Ricos 2014 biological goals - CLIA didn't directly regulate in in 1992).  But if we use the CLIA goals from the 21st century, the picture changes:

The shift is real and significant. From a benchmark that puts nearly everything at 6 sigma, the picture shirts so that nothing is in the bull's-eye. Instead of being able to get rid of most "Westgard Rules", many of these assays require all the "Westgard Rules".

For the electrolytes, the Atellica posters measure performance across the reportable range. For potassium, CLIA tightened the performance goal from 0.5 mmol/L to 0.3 mmol/L. This "before" and "after" are illustrated below:

That looks great. Near the low end of the range, Six Sigma performance. Along the upper part of the range, Four Sigma. Sadly, this is only great performance when the 1992 goals are used. If CLIA 2024 goals are used instead, the picture changes:

The very lowest part of the range is still in the Bull's-Eye.But now the upper range is performing at 2 sigma and less than 2 sigma.

Creatinine also went through a significant change: from 0.3 mg/dL or 15% (whichever is greater) in the 1992 goals, to 0.2 mg/dL or 10% (whichever is greater) in the 2024 goals.

Using 1992 standards, the Atellica creatinine method is hitting the Bull's-Eye across its entire reportable range. However, we're not in 1992.

Now just under half the performance is only considered minimally acceptable.

Some goals did not change under the new CLIA 2024 regulations. But they remain very challenging still. For instance, sodium and urea nitrogen:

In fairness, sodium has been a challenging assay for most instrumentation, and thus it's not a surprise that even the latest generation here cannot hit the CLIA targets. Nevertheless, it does not inspire enthusiasm to find the new generation of instruments here is promising to require a very large investment in Westgard Rules and controls.

Conclusion

CLIA 2024 is a big change. There is a big difference from the 1992 CLIA-based Sigma metrics published by Siemens at the AACC/ADLM meeting, and the correct Sigma metrics that should have been calculated against the impending 2024 goals.

Having instruments that benchmark themselves using goals from the previous century does not make it easier for laboratories to understand which box to purchase. Buying an instrument based on 1992 benchmarks only works if you intend to take the instrument into a time machine and go back to the laboratory of the past to operate it.