Trends

• What's Legal under CLIA?
• Does autoverification without QC assure quality?
• Why not do it right?
• Who will become responsible?
• Minimum compliance is not good enough for our patients
• References

A couple of matters in the news caught my attention over the holidays. The November 2003 issue of Clinical Laboratory Science provided an interesting discussion of “autoverification”, which was defined as a “set of logic or computer rules that are designed to mimic exactly what laboratorians would do it they were manually verifying test results – without errors” [1]. Dr. Larry Crolla and I just published an article on the “Evaluation of Rule-based Autoverification Protocols” in the September/October 2003 issue of Clinical Laboratory Management Review [2]. You can imagine my interest in looking at the rules being used in real laboratory operation vs the evaluation guidelines we have developed.

Another matter that has achieved particular attention in the Wisconsin newspapers is the mutual fund scandal. Strong Financial happens to be a Wisconsin corporation that has implicated in this scandal. Strong holds the contract the State of Wisconsin “Edvest” business, which is the official state program that allows parents to invest money for their children’s college education. As you may know, a number of mutual funds have been implicated in after hours trading, something that isn’t necessarily illegal, but is still considered to be wrong. The December 5, 2003, Capital Times (a Madison newspaper) described the issue this way:

“Investigators looking at the mutual fun industry are focusing on so-called market-timing transactions, in which short-term, in-and-out trades are used to capitalize on market-moving news. The process is not illegal, but many fund companies – including Strong – have policies against it because it increases costs and hurts long-term shareholders. Regulators have indicated that it is fraudulent for a fund to allow selective market-timing without disclosing that to shareholders.”

You may be wondering why I mention these seemingly unrelated topics here. Actually, there is a relationship – what’s legal is not the same as what’s right!

What’s legal under CLIA?

The autoverification process being discussed in the Clin Lab News article failed to include a basic rule that internal QC must be part of the ongoing validation of test results. It appears that a common mode of operation in laboratories today is to setup the analytical system, calibrate it, run fresh controls, rerun controls as necessary, and then do whatever else that is necessary to force the control results to be in. Once that’s accomplished, they proceed to autoverify for the rest of the day.

I hope that practice doesn’t represent “exactly what laboratorians would do if they were manually verifying test results – without errors.” However, manufacturers tell me that is exactly what laboratories are doing and exactly how laboratories want autoverification to function. Even if the manufacturers provide the capability for ongoing QC monitoring, laboratories don’t or won’t use that capability.

In my opinion, implementation of autoverification in this manner is a step in the wrong direction, a level down rather than up! And, under CLIA, that practice is legal because compliance only requires running 2 controls every 24 hours! If your only interest is to be in compliance with CLIA, you’re good to go. Just be sure to advertise your laboratory as a minimum compliance business.

Does autoverification without QC assure quality?

The particular rules being included in the autoverification protocols being discussed in the Clin Lab News article are mainly of the following type:

• Is the test result within the normal range, or a similarly defined interval where interpretation is not expected to be critical?
  
• Is the test result within a specified range for a delta check?
  
• Is there any inconsistency between this test result and another test result?
  

The quality of test results in the normal range does matter! Look at what’s happening with the new ADA guidelines. The latest suggestion is to drop the glucose cutoff even further, to 100 mg/dL. That’s within the expected reference range for glucose. Even normal results need to be correct given today’s aggressive interpretation of laboratory tests.

Patient data checking rules are not as powerful as statistical QC rules! These data checking rules are logical, but they’re not very sensitive for detecting errors. There certainly is anecdotal information to justify these kinds of rules, but they generally haven’t been carefully evaluated in a manner that permits comparison to statistical QC. Work that I’ve been involved with on patient data algorithms has shown the need to use a group of patient samples to provide reasonable error detection. For example, the mean anion gap for a group of 8 patients is much better for monitoring test performance than individual anion gaps [3]. It is also much better to use of an “average of normals” (AoN) for a group of patient results, rather than compare individual patient results to a reference range. AoN algorithms, in the best case application, require a minimum of 10 to 20 samples, but more often will require a hundred or more [4,5].

Lab-specific autoverification rules have not been well studied! The most extensive application of autoverification is found in Europe, where the approach is termed “medical validation.” Two programs, VALAB and LabRespond, have been studied and their performance documented in the literature [6,7]. These programs employ a series of rules that have been developed and tested by professional groups, therefore they represent the collective wisdom of many laboratories, rather than rules selected by an individual laboratory, which is the practice emerging in the US. It should also be noted that statistical QC is part of the overall verification process, e.g., the third level of checking in LabRespond, prior to checking by patient data algorithms.

We invite you to read the rest of this article

This complete article and many more essays can be found in the Nothing but the Truth about Quality manual, which is available in our online store. You can also download the Table of Contents and additional chapters here.