Tools, Technologies and Training for Healthcare Laboratories

The Five Bests of Six Sigma: A Radical Approach to Choosing Instruments

Isn’t Six Sigma just common sense? Do we really need techniques, or equations, or graphs, to tell us that we should get the best instrument and do the best QC?

A Radical Idea: The Five Bests of Six Sigma

Sten Westgard, MS
September 2019

In a recent lecture, I had a very perceptive question. For all we talk about Six Sigma quality and Westgard Rules, isn’t our point simply to say, “Buy the best instrument with the best quality, do the best QC, and you will get the best performance and deliver the best patient results?”  In other words, for all the tools and techniques we present, aren’t we really just saying something that is common sense?

I would argue that, especially in today’s world, as Voltaire put it, common sense is not so common. Indeed, it's almost radical, and it can be increasingly difficult to do, if not rare.

Let me break down the statement into pieces, and explain why what we propose is actually a kind of radical common sense.

The Best Instrument? Is it common sense which one is best?

Is the best instrument the one that has the best (cheapest) price? Or the best (fastest) throughput? Or the best (smallest) footprint? Or the best (“Free”) automation track? Or the best scientific (most traceable) assays? Or all of the above? Or none of the above?

First, everyone’s “best” is a unique combination of factors that fit their context and serve their patients. Second, every manufacturer argues their methods, instruments, and solutions are best. Third, if the decision is (as it increasingly is) driven by administration and executives outside the laboratory, the quality piece is often assumed, not assessed. If instruments are commodities, quality is assumed to be the same, and price is all that matters.

So, yes, it’s a radical solution to determine that the Sigma-metric (analytical) quality of an instrument is a factor. That quality is not a commodity. That the manufacturers are unequal in the offerings.

The best instrument is still relative – no one can afford to pursue the very best analytical quality without regard to price and cost – but neither can we afford to make decisions only on the low bid.

The Best Quality? Is it common sense which method is best?

Again, every manufacturer claims the best quality. But at least we are in an area of focus, where we can concentrate on analytical performance of imprecision (CV) and bias (inaccuracy, trueness). Simply taking the lowest CV and lowest bias is not the only strategy. Sometimes even low CV and low bias may not be good enough – we might find out that, in the context of medical decision-making, the performance still isn’t acceptable.

We can judge the suitability of a method by using allowable total error (TEa) and the analytical Sigma-metric. Which we detail in many other articles and case studies on the website. But for all of our work, approximately 20 years of effort and promotion, it's not a common practice. Our survey of Six Sigma practices in 2018 found that while 90% of labs know about Six SIgma, less than a quarter of them practice it. And really only about 15% of labs worldwide truly use Sigma-metrics in their benchmarking and QC.

There’s an increasing emphasis on measurement uncertainty (mu) over analytical sigma-metrics. This is a trend being pushed by the metrologists and re-enforced by ISO 15189 guidance, without any proof or demonstration of practical utility for analytical quality management.

Let’s be blunt: mu won’t help you make instrument decisions. Measurement uncertainty, when expressed, doesn’t give you an idea of acceptability. It just gives you more uncertainty.

So, yes, it’s a radical proposal to assess analytical quality with a Sigma-metric and use that to determine which method is most attractive for your lab and most appropriate for your patients.

The best QC: Is it commonly known what good QC even looks like?

We know it all too well. The common practices of QC in laboratories around the world are the opposite of common sense – it’s closer to nonsense. The use of antiquated (read: 2 SD) control limits, the endless hours of repeating and recalibrations to force control values back “in”, the tortured control limits adopted that stretch so wide a truck can drive through them. Our own survey, and a recent survey of the “top 21” large academic medical centers in the US confirm that wasteful QC practices are the norm, not the exception. QC tends to fluctuate between the hyperactive (alarms going off multiple times a day) and the comatose (no outliers for weeks). Neither is a good sign.

It’s not commonly understood that the best QC is not “all 2 SD” nor is it “all Westgard Rules.” A flexible, dynamic approach is more efficient and effective. The best QC uses just enough rules to detect medically important errors, but no more rules (or control) than necessary. By making sure we don’t over-control, as well making sure we don’t under-control, each test, we can reach the real “best” in QC practices.

So, yes, it’s a radical propose to undo all the bad habits of QCs in laboratories around the world and move them to a customized, data-driven QC system.

The best performance: Is it commonly known what good performance looks like?

Uptime, turnaround time, time on the track, most of the measurements we obsess about on operations are driven by time. We make the assumption that the results we are delivering faster and faster are also correct, which is sometimes a baseless assumption.
Elements of operational performance that every bench tech suffers but are less scrutinized: how many out-of-control events are occurring, how many trouble-shooting episodes, how many patients are being retested because the initial result was inconclusive, indeterminate or nonsensical?

The best performance reduces the number of rules, the number of controls, and even (potentially) the frequency of controls to its absolute minimum. Which drives down the number of outliers, which drives down the hours spent chasing these self-induced out-of-control events, which drives down control and calibration costs, which drives down reagent burn, which delivers reduced hard dollar costs and reduced soft dollar labor costs.

So yes, it's still radical to measure analytical quality and appropriate change the practice of QC, troubleshooting, and reporting.

The best results: is it common sense to choose the faster result over the accurate result?

Again, turnaround time is the most commonly tracked metric on results, from the laboratory perspective. It’s far less common to use the analytical sigma-metric, and estimate out of a million reportable results, just how many of those are defects (false positives, false negatives). At Six Sigma, we expect less than 4 defects per million reportable results.

Even less common are health economics outcome research studies (HEOR) that connect the quality of the reported result to the patient outcomes like misdiagnosis rates (under- or over-treatment), patient care costs that result from that, and impact to quality-adjusted life years (QUALYs). These are difficult studies to conduct, involving significant investment in modeling, statistics, and simulation. But the results we’ve seen so far show that better analytical quality leads to better outcomes, longer lifespan, and cheaper care.

Yes, that’s a radical idea. To take the lab test, from which we purportedly make 70% of our medical decisions, and invest in better analytical quality. When we make that commitment to the very core of the test result, it’s one of the best returns on investment.

The Five Bests – a radical Six Sigma approach

When you put together all of these “bests”, I sadly have to admit that very few organizations keep all of these factors in mind when they make their instrument decisions. While I have been fortunate to work with labs that do, and diagnostic manufacturers that do, and vendors that do, and even purchasing organizations that do, I know my experiences represent a minority of how decisions are being made in the world.

Too often, what we see in labs making decisions is that they don’t know, can’t articulate, the impact of their instrument choices on patient outcomes. And that the overwhelming temptation is to pretend

  1. Quality doesn’t matter, since the clinicians “interpret” the results into the correct clinical decision, regardless of what instrument is chosen; or
  2. Quality is a commodity, therefore it is the same across some/most/all instrument platforms; and/or
  3. Buying the cheapest, fastest instrument is all that is necessary – and by cheap, it’s the one with the lowest up front prices, even if over the long run, it’s more expensive to operate.

This is the common nonsense that is being practiced by many laboratories worldwide. Once we can reduce that to a minority of decisions and labs, then, yes, what we’re talking about at Westgard, what we’re presenting to audiences globally, will truly by common, and those decisions will truly make sense.