QC Design

Why is QC so starkly different in hematology? The ICSH has reviewed current practices, found them wanting, and has made recommendations for improvement.

Learning to see the real QC in Hematology

April 2024
Sten Westgard, MS

There’s a strange paradox one can experience in laboratories. On any given day, you walk over to the chemistry section, and the question is, What is out of control, not If they are out of control. Then if you wander over to the hematology section, you hear crickets, you see tumbleweeds rolling by on their Levey-Jennings charts. There are no out of control events that day or many previous days. Are hematology instruments perfect? Or is there something else happening with QC in hematology?

Two important new papers from the ICSH have reviewed the current state of hematology QC and – after digesting those findings – have made recommendations for labs and manufacturers. They have identified key problems and proposed some solutions.

ICSH review of internal quality control policy for blood cell counters, McCafferty R, Cembrowski G, de al Salle B, Peng M, Urrechaga E. Int J Lab Hematol 2024:1-11 DOI: 10.1111/ijlh.14220

ICSH guidance for internal quality control policy for blood cell counters. Cembrowski G, de al Salle B, Peng M, Urrechaga E. Int J Lab Hematol 2024:46:227-233 DOI: 10.1111/ijlh.14212

A disappointing Confirmation about Hematology QC

As gratifying as it would be to have hematology instruments that are error-free, the reality is that the instruments are imperfect, but the hematology QC traditions make the problems much harder to see. The control limits that are recommended by manufacturers – sometimes strongly recommended – are leaving labs blind.

The ICSH survey found that many control rules and control limits in use are egregiously wide. Not just ‘drive a truck through it’ limits, but sometimes even reaching the ‘steer an aircraft carrier though it’ limits. [please note: those are my quotations, not theirs]. Take a look at the figure below from the UK on hemoglobin. For hemoglobin, more than 72% of the surveyed labs set their control limits at 3 SD or wider. 42% of those labs set their control limits at 4 SD or wider. One outlier laboratory set their control limits wider than 10 SD.

This presence of 10 SD limits is replicated elsewhere in MCV, and MCHC. “HCT needs to shift by more than 5… SD to indicate an error (1-6s control rule. HCT is a noisier analyte than HGB and strangely, the limits for detecting error in HCT have been expanded compared to HGB.”

[Let's be real clear: 10 SD limits are a bad choice, not good laboratory practice.

Back when we were developing QC Validator and EZ Rules technology, the widest the control limits ever got were 6 SD, and the 1:6s control rule was the widest control rule we ever assessed for probability of error detection and probability of false rejection. It was unimaginable that labs would ever seriously contemplate wider limits.

Even with the most advanced QC desig tool we have now, the Westgard Sigma Rules, we do not recommend wider limits than 3 to 3.5 SD for even the best method. For a Six Sigma method, world class quality, the simplest recommendation is therefore the 1:3s control rule, not something wider. We haven't even tried to calculate how high the Sigma metric would need to be in order to justify 7, 8, 9 or wider control limits.]

Globally, over 42.4% of laboratories use such manufacturer-supplied control limits, while just slightly more (42.9%) establish their own mean, standard deviation and control limits. When assessed by country, the practices vary even more, with 60% of labs surveyed in China using their own limits, while 87% of labs surveyed in Ireland and Spain using manufacturer limits.

From a cynical perspective, wide limits are a good deal for manufacturers, who don’t have to worry about reagent lot changes and reduce the number of technical support calls and complaints from their customers. It’s not so great a deal for laboratories, who may enjoy the illusion of outlier-free instrument operation, but may also face the ire of those who may be missed or mis-diagnosed due to uncaught errors in the results. And it’s not a good deal at all for patients, whose health might be severely impacted by those errors.

Three Important Improvements and one minor side-step

ICSH digested the unpalatable results of their survey and attempted to craft some solutions:

“The diagnostic laboratory should calculate the numerical differences from the observed mean or target value in standard deviations, of the action limits they are using. They should then tighten those limits if the parameter of the FBC/CBC is considered clinically important but the limits in use are unlikely to detect analyzer error.”

This implies some ability to assess the error detection of control procedures. If you don’t know the error detection capability of a 1:5s, 1:6s, 1:7s, etc. control rule, you shouldn’t use that rule. And it also follows that error detection will vary depending on the performance of the method itself. Your specific operation of the method may require you to use “Westgard Rules”, while another lab, if they’re doing better, may only need that 1:5s rule.

“[ed. The lab] should choose whether to apply their own calculated action limits [ed. this means a control rule] as prescribed by CLSI H26-A2, or to tighten the limits provided by the manufacturer.”

This recommendation encourages laboratories not to simply accept the advice on control rules given by the vendor. Calculate and formulate your own QC rule, or tighten what’s been given by the vendor.

In order to further empower the laboratory’s individual decision-making, the ICSH recommends forcing the manufacturers and vendors to be less coy about the limits they recommend:

“All suppliers of commercial stabilized controls, whether the cell counter manufacturer or supplier of ‘independent third-party controls’ should state clearly with product inserts, how they consider the product can or should be used, to promote consistency of practice around the world.”

When manufacturers in the hematology field provide means and ranges, they may state in fine print – or only upon being queried – that these ranges are only meant to be used initially in the verification process of the control, not for long-term use. This is a don’t ask, don’t tell, kind of policy. The manufacturer gives a range but doesn’t explicitly tell the lab it’s only for initial use; the laboratory doesn’t ask how long the range should be used.

“[I]f a cell counter manufacturer does not recommend that the target and limit values supplied with the assay sheet be used routinely… or whether they should be verified before use by the diagnostic laboratory, this should be clearly stated in the product insert.”

The last recommendation for manufacturers is to convert their ranges into something more that makes clear just how wide they are:

“We also propose that the manufacturer should clearly indicate their provided action limits [ed. Again, this means control rule] as multiples of SD for their common analyzers in order to further inform their users.”

Here I admit a bit of confusion – whose SD? When a manufacturer makes their recommendation, they can know their group SD, but cannot precisely know what each individual laboratory’s SD is. If the manufacturer simply states that their range represents 2 SD of their group, that still requires the conversion of the group SD into the individual laboratory’s SD. A group 2 SD might actually mean 4 SD of the individual laboratory, so the problem persists.

Third Party Controls no more?

ISO states that “Independent third-party control materials should be considered, either instead of, or in addition to, any control materials supplied by the reagent or instrument manufacturer.” It’s part of the typical QC dogma that controls should be from a different provider than the manufacturer. But the area of hematology has fewer options, with only 2 major independent control vendors, who often bundle those same controls with the manufacturer. In the review, ICSH calls them “so-called independent third-party controls.”

The danger is that the manufacturer, whether they actually make the controls or they control the package inserts, has a clear incentive to make those controls always fall “in” when they are used by their customers. Independent controls eliminates that danger.

ICSH asserts “IQC products from third-party suppliers other than the cell counter manufacturers may not be inherently superior. Therefore, the laboratory may use third-party controls, if this helps control costs, but should not be obliged to use them is this adds no additional quality.”

Two points on this: there has to be a mechanism to determine the “additional quality.” Is unbiased advice “additional quality” enough? Or is this just a qualitative, unconfirmable judgment? If you simply 'believe' the third-party controls aren’t helpful, is that enough to switch back to 'free' manufacturer-provided controls? Can you compare the third-party controls to the manufacturer-provided controls and reasonably replicate the same results? It seems like an easy experiment to conduct – and the laboratory should use their leverage to force the manufacturer or the control vendor to provide those controls for free. One most also note that the cost savings from the manufacturer-provided controls are simply an illusion - the cost of the controls is borne by the instrument costs. Labs always pay for controls, but clever manufacturers disguise the cost.

Yet there is a new angle here, though. If indeed the control materials from 1st and 3rd providers are essentially the same “juice,” then the only thing separating manufacturer from independent provider is the advice and recommendations that come on the package insert. So if you throw away the manufacturer’s recommendations, you can use the controls themselves as if they were independent.

The ICSH review and recommendations are more numerous than what we’ve discussed here, and worth reading directly. But for QC, the overall message is: today's hematology QC is not working correctly. However, if manufacturers, vendors and laboratories heed this new ICSH advice, QC can work a lot better in the future.