Questions
Questions from the 2024 CLIA GAMECHANGE webinar
After record-breaking attendance at the CLIA 2024 GAMECHANGE webinar, there were so many questions, we couldn't answer them all. Here we share some of the questions we didn't have time to answer during the broadcast.
Questions (and Answers!) from the GAMECHANGE CLIA 2024 webinar
June 2024
Sten Westgard, MS
More than 3,000 lab professionals registered for the June Westgard webinar on CLIA 2024 proficiency testing changes (GAMECHANGE). If you didn't get a chance to attend, you can still sign up for the on-demand version of the webinar. As you can imagine, an audience of that size generated dozens and dozens of questions, many of which we were unable to answer in the time given to the webinar. So here is our follow-up, addressing the most common and relevant questions.
Q: Where can see see the list of all of these new CLIA proficiency testing changes? [paraphrased]
Certainly you can get it straight from the horse's [CLIA] mouth, and that's located here:
However, if you're not a fan of reading regulations in all their arcane terminology, there's a simplified table here:
https://westgard.com/clia-a-quality/847-2019-clia-requirements.html
And if you want to see these goals in comparison with other goals from around the world, check out our consolidated pages here:
https://westgard.com/clia-a-quality/quality-requirements/802-hematology-goals.html
These pages are also useful in answering a follow-up question: What goals are there for analytes (still) not directly regulated by CLIA?
CLIA is always going to be behind in directly regulating analytes, indeed any EQA or PT program will face that problem, as new tests continue to emerge and new clinical uses of existing tests continue to evolve. And there continue to be blind spots in testing, where EQA and PT programs fear to specify. For example, viral load testing in general lacks concrete, quantitative performance specifications, and almost no program provides it. There will always be areas where the laboratory director must use their judgment to set local, relevant performance requirements.
Q: Do laboratories outside the US have to follow CLIA's new goals?
CLIA's new goals are mandatory for laboratories operating within the US. However, there are different EQA goals and systems outside the US. International labs that seek the coveted CAP or JCI accreditation, however, must also adhere to the CLIA goals. And further, some countries that did not set their own performance specifications, will follow CLIA goals out of convenience (some provinces of Canada do this, for example). Overall, the CLIA goals are effectively a global standard, in rivalry only to the EFLM biological variation database goals.
Q: How have diagnostic manufacturers responded to these new goals? [paraphrased]
Over the past two years, since July 2022, we have been trying to discern the response of the major diagnostic manufacturers. In several cases, it appears that they - or at least their local representatives - were not aware of the changes, and/or thought the changes would have no impact on the grading of their instrument performance. Even as late as mid-2023, some manufacturers continued to put out performance evaluations using the now-obsolete CLIA 1992 goals. More recently, in 2024, several diagnostic manufacturers are benchmarking their methods with the new 2024 goals, while others avoid using performance goals entirely.
We're in a strange period, where we have new goals being applied to old instruments. For some diagnostic manufacturers, they were already aiming toward performance better than CLIA 1992, so the 2024 update won't have as much impact on them. For other diagnostic manufacturers, who may have been "coasting" their instruments on the CLIA 1992 goals, it may take a long time to catch up. Typically it takes about 10 years to bring a new instrument to market, so there will be a set of labs that suffer while they await better performance, unless they switch away entirely.
This latest CLIA change exposes the lie to one of the pervasive myths of laboratory diagnostics: that all instruments perform the same, and you just pick one based on price. Often the very cheapest instruments provide those upfront cost savings at a steep price: inferior quality of results. As we have encouraged for decades, laboratories need to intensely scrutinize the analytical performance of the instruments they consider for purchase. A one-time savings in price that comes with a long-term cost of poor performance is no bargain at all.
Q: "Since proficiency testing grading tends to be pased on peer group consensus, would another indicator of future PT performance be to apply the new CLIA limits to past proficiency testing statistics? An instrument, such as Siemens, may be of lower sigma but could continue to compare with it's [sic] peers and still pass the PT."
The first part of the questino is great: go back to your old PT results, impose the new goals, and see how many more times you'd fail (or not). Indeed, some of the PT providers have this feature available to their members (but won't do the analysis as a courtesy to all participants, mind you). The second part of the question, well, this is a rather unfortunate admission: that Siemens instruments perform more poorly. But poorly performing instruments that are consistently poor along with their peers is not a saving grace. In the old CLIA 1992 goals, there were many analytes that were indeed governed by "3 SD" performance specifications, meaning your were judged by the company you kept, namely 3 SD of the peer group, not the specific quality you exhibited. Think of it this way, those old goals were like playing musical chairs with 100 people and 99 chairs. In those cases, you could get away with poor performance, because as long as all of your peers performed badly, you'd still be within 3 SD of them. But those "3 SD" goals are being eliminated in many cases, reverting instead to specific quantative goals like 20%, 10%, etc. Now the game of musical chairs changes, because you might have 100 people, but only 60 chairs. The worse your peer group was in the past, the worse wake-up call you're going to have on January 1st, 2025.
A longer discussion on the issue of 3 SD goals can be found here: https://westgard.com/essays/guest-essay/396-guest39.html
That provides a representative sample of the questions from GAMECHANGE. Remember, you can watch the webinar on-demand.
And if you have any questions about CLIA 2024, please send them in. For over 30 years, Westgard QC has been here to give you the best answers.