Tools, Technologies and Training for Healthcare Laboratories
  2. Home
  3. Lessons
  4. Quality Management
  5. Total Quality Control Strategies

Quality Management

Total Quality Control Strategies

You've got some methods in your laboratory that you don't have to worry about (right?). Then there are some that from time to time, have "issues." And then there are the persistently difficult methods, where it's always out-of-control and you don't know where the problem lies. And you have to make sure your scarce personnel are making all these methods work? Here's a new way to do it: a TQC strategy. Using TQC strategies, you'll know what to do for every method, when to do it, and when to move on.

Visualize Hagar the Horrible standing waist deep in water, gazing at a map, and asking Lucky Eddy "why didn't you indicate on the map that this was deep water." Lucky Eddy replies, "I ran out of blue crayon". Well, as you work through the QC planning process and find yourself gazing at an OPSpecs map, you may also find occasions when you are in deep water. This lesson is the "blue crayon" to help you identify hazardous situations where analytical quality is difficult to manage and to give you some ideas on what to do in these situations.

Need for a TQC Strategy

Current QC practices tend towards a uniform or average quality control system that is applied to all testing processes. However, a testing process with better performance than average should require less quality control and one with worst performance than average should require more quality control. In this era when the quality and cost of healthcare is of great concern, testing processes should be optimized for cost-effective operation, test by test, based on the quality required and the performance observed.

The cost-effective operation of laboratory testing processes depends on formulating quality control strategies that are appropriate for the quality requirements and performance characteristics of each testing process [1]. US government regulations for laboratory testing (CLIA, Clinical Laboratory Improvement Act) [2-3] define quality control requirements that include method performance specifications, statistical quality control, preventive maintenance, instrument function checks, and method performance tests. These CLIA rules may be viewed as separate requirements for individual components of Quality Control (QC), or as a requirement for developing a Total Quality Control (TQC) strategy that incorporates these components in a manner appropriate for controlling individual testing processes. The latter view would seem to be more desirable for assuring the quality of laboratory testing because of the need to individualize the QC designs for the many different analytical methods for performing those tests.

Components of a TQC Strategy

The responsibility for establishing a TQC strategy initially belongs to the manufacturers of medical testing systems, devices, or kits. When a manufacturer's QC instructions have by cleared by FDA as meeting CLIA requirements for quality control, the CLIA rules require that a laboratory does the following:

  • (a) "demonstrate that, prior to reporting patient test results, it can obtain the performance specifications for accuracy, precision, and reportable range of patient test results, comparable to those established by the manufacturer [2, p. 5230, par. 493.1213(b)(1)],
  • (b) "perform maintenance as defined by the manufacturer and with at least the frequency specified by the manufacturer," [2, p. 5231, par. 493.1215(a)(i)],
  • (c) "perform function checks as defined by the manufacturer and with at least the frequency specified by the manufacturer" [2, p. 5231, par. 493.1215(b)(i)],
  • (d) "follow the manufacturer's instructions for calibration and calibration verification procedures using calibration materials specified by the manufacturer" [2, p. 5231, par. 493.1217(a)], and
  • (e) "follow the manufacturer's instructions for control procedures" [2, p. 5232, par. 493.1218(a)].

For a test method whose QC instructions have not been cleared by the FDA, the laboratory itself assumes responsibility for formulating an appropriate strategy for quality control that includes these same components. Thus, both manufacturers and laboratories need a rational approach for developing TQC strategies for a wide variety of laboratory tests and analytical testing processes.

General Guidelines for Formulating a TQC Strategy

There is little guidance on how to do this in the current laboratory literature. NCCLS is currently working on a document on "Quality Management for Unit Use Testing" that describes a total quality management system that will help identify potential error sources along with appropriate quality monitors. The NCCLS document will deal with the total testing process which includes pre-analytical, analytical, and post-analytical factors. Our discussion here focuses on a TQC system for analytical factors and analytical quality, but should still help you understand the concept and give you some good ideas on how to do this.

TqchartThe starting point for formulating an analytical TQC strategy is the QC planning process that is used to select a statistical QC procedure and assess its error detection and false rejection characteristics. See one of our example applications to review this planning process. After the error detection capability has been assessed, the TQC strategy can be formulated as suggested in the accompanying table, where the number of x's represents the weight or emphasis on that particular component.

The initial objective is to select a QC procedure that will provide 90% detection of the critical systematic error while maintaining a false rejection rate of 5% or less. If the statistical QC procedure can provide a 90% chance of detecting the critical systematic error, the TQC strategy should be to depend on statistical QC as the primary component. On the other hand, if the error detection is less than 90%, the TQC strategy should emphasize statistical QC, method improvement, and error prevention. If error detection is less than 50%, the TQC strategy should place a high priority on method improvement and error prevention.

Flowchart for Formulating a TQC strategy

Tqcflow1The QC planning process is applied to assess the probability of detecting the critical systematic error. The analytical process is then classifed into one of three groups based on the probability of error detection (Ped): high error detection when Ped>90%, moderate when Ped= 50-90%, and low when Ped<50%. If the process has high error detection, then the emphasis is on minimizing the costs of statistical and non-statistical QC. If the process has moderate error detection, then the emphasis is on maximizing statistical and non-statistical components, as well as improving measurement performance. If the process has low error detection, the efforts also include optimizing QC for process stability, improving the skills of the analysts, and adding patient data QC. In all cases, the final step is to document the QC system.

Step-by-Step Guidelines

Minimize the cost of statistical QC: Use as few control measurements as needed, reducing N to a minimum of 2 whenever possible. Use single-rule rather than multi-rule procedures. Widen control limits for single-rule procedures up to 3.5s when Delta2SEcrit is large. Aim for 1% or lower false rejection. Increase run length to maximize test yield, i.e., the ratio of patient samples to control and calibration samples.

Minimize cost of non-statistical QC Recognize the limitations of control materials and their matrices when minimizing non-statistical QC. Weigh the clinical needs and risks carefully. Then identify the minimum frequency of system function checks, performance validation tests, and preventive maintenance, as required by regulations, manufacturer's instructions, and good laboratory practice.

Maximize error detection Increase N from a minimum of 2 up to at least 4 controls per run. Increase run length as a way of increasing N, being careful to satisfy the turnaround time requirements for the test. Narrow the control limits and tolerate up to 5% false rejections. Change from single-rule to multi-rule QC procedures. Look-back at control data in previous runs to increase N by using past control data. Implement multi-stage QC procedures that have startup designs with higher N's and/or more sensitive control rules to maximize error detection, then switch to a monitor design having lower N and/or less sensitive control rules that minimize false rejections. Switch back and forth between the startup and monitor designs as necessary.

Maximize non-statistical QC Perform the preventive maintenance, calibration, instrument checks, and performance verification tests that are required by CLIA, recommended by the manufacturer, and appropriate for the susceptibility of the method and the clinical application of the test.

Improve method performance Reduce analytical bias by selecting appropriate standards, calibrating properly, and by selecting proper comparision groups in proficiency testing surveys. Reduce imprecision by identifying and minimizing the major component of variance, standardizing operator techniques, and mechanizing manual steps in the process. Reduce frequency of errors by identifying and eliminating sources of problems, optimizing the preventive maintenance schedule, increasing function checks and performance verification tests, reducing operator variables, and increasing operating training and expertise. When necessary, change measurement procedures or analytic systems to obtain better accuracy, precision, and stability.

Optimize QC for process stability Document the frequency of errors by careful study of the analytical process. Define an acceptable defect rate and determine the error detection necessary to maintain that defect rate. In general, a 50% error detection rate will be satisfactory for stable processes that have <2% frequency of errors and even a 25% detection rate may be sufficient for extremely stable processes that have <1% frequency of errors. The expected defect rate, Defectexp, can be estimated from the following equation:

Defectexp = f(Defectmax)(1-Ped)

where f is the frequency of analytical runs having medically important errors, Defectmax is the maximum defect rate allowed before an analytical run is rejected by the QC procedure (which is related to the choice of z-value in the quality-planning model), and Ped is the probability of rejecting a run having a medically important errors. In practice, we use the probability of detecting the critical systematic error as the estimate for predicting defect rate.

Deploy skilled analysts Assign highly skilled and experienced analysts to testing processes that are problematic and difficult to control. Provide thorough in-service training. Increase technical skills and experience. Improve problem solving abilities. Improve statistical skills for method validation and quality control.

Add patient data QC Perform between system comparisons on patient samples. Check patient data with consistency algorithms, such as delta checks, anion gap, etc. Utilize population statistics, such as mean of normals, Bull's algorithm, etc. Perform clinical correlations to check test results with patient diagnosis and condition.

Document TQC strategy Document the QC acceptability criteria for assessing the control status of an analytical run. Describe the expected performance characteristics using power function graphs, critical-error graphs, or OPSpecs charts. Establish the schedule for performing non-statistical QC components and document the procedures for performing those tests. Identify methods that problematic and in need of improvement. Set priorities for improvement of existing methods and the development and/or acquisition of new methods.

Example Application for Multitest Chemistry Analyzer

Studies on a multi-test chemistry analyzer (4-5) provide examples of individual tests that fit these three categories. Analytical quality requirements were defined in terms of an allowable total error and were generally as demanding, and occasionally, more demanding than the CLIA proficiency testing (PT) criteria for acceptability, but it should be noted that these studies pre-dated the publication of CLIA PT criteria.

High error detection, 90% or greater for the critical systematic errors, could be achieved using a 13.5s control rule with N=2 for the following tests: sodium (Delta2SEcrit=4.27s), potassium (Delta2SEcrit=6.99s), glucose (Delta2SEcrit=5.02s), urea nitrogen (Delta2SEcrit=5.87s), creatinine (Delta2SEcrit=8.35s), phosphorus (Delta2SEcrit=6.16s), uric acid (Delta2SEcrit=7.44s), cholesterol (Delta2SEcrit=5.76s), total protein (Delta2SEcrit=4.87s), total bilirubin (Delta2SEcrit=7.44s), GGT (Delta2SEcrit=6.90s), ALP (Delta2SEcrit=6.90s), AST (Delta2SEcrit=5.02s), and LD (Delta2SEcrit=5.02s). For these thirteen tests, control limits were set at 3.5s rather than 3.0s to reduce false rejections to essentially zero. In addition to these tests, 90% error detection could also be achieved for albumin (Delta2SEcrit=3.04s) by use of a 12.5s control rule with N=2. A false rejection rate of 2-3% could be tolerated in exchange for increasing the error detection rate from about 50% for 3.5s control limits to 90% for 2.5s control limits. For all these tests, the TQC strategy is to rely on statistical QC and perform the minimum requirements for non-statistical QC components.

Moderate error detection was achieved for chloride (SEcrit=2.20s) and total CO2 (SEcrit=2.35s). With careful maintenance, we observed 1.6% problem runs for chloride and 0.0% for total CO2. Use of a 12.5s control rule with N=2 provided sufficient error detection (66% for chloride, 72% for total CO2) to achieve a defect rate of 0.027% (0.016*0.05*0.34) or approximately 3 defects per 10,000 tests for chloride and less than that for total CO2. Use of a multi-rule QC procedure would provide some improvement in error detection and should achieve an even lower defect rate. For these two tests, the TQC strategy is to balance statistical QC with non-statistical QC components.

Low error detection was observed for calcium (SEcrit approx. 1.3s), where there was only about a 30-40% chance of detecting the critical systematic error. To utilize this measurement procedure for routine service, two calcium channels were set up to provide duplicate measurements on all patient specimens. These patient duplicates were then checked by a special QC program and averaged to provide a more precise test result. With careful maintenance, the frequency of problem runs for calcium was only 0.5%, thus a defect rate of 0.016% (0.005*0.05*0.65), or approximately 2 per 10,000, could be achieved even with the low error detection from statistical QC. For this test, the TQC strategy is to improve method performance by making duplicate measurements, then to maintain the method agressively to prevent problems from occurring.

References

  1. Burnett RW, Westgard JO. Selection of measurement and control procedures to satisfy the Health Care Financing Administration requirements and provide cost-effective operation. Arch Pathol Lab Med 1992;116:777-80.
  2. Health Care Financing Administration (HCFA) and Public Health Service (PHS), US Dept of Health and Human Services (HHS). Medicare, Medicaid and CLIA Programs. Regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA) and Clinical Laboratory Improvement Act program fee collection. Fed Regist 1993;58:5215- 37.
  3. U.S. Dept. of Health and Human Services. Medicare, Medicaid, and CLIA Programs: regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA). Final Rule. Fed Regist 1992;57:7002-186.
  4. Koch DD, Oryall JJ, Quam EF, Feldbruegge DH, Dowd DE, Barry PL, Westgard JO. Selection of medically useful quality-control procedures for individual tests done in a mujltitest analytical system. Clin Chem 1990;36:230-3.
  5. Westgard JO, Oryall JJ, Koch DD. Predicting effects of quality-control practices on the cost-effective operation of a stable multitest analytical system. Clin Chem 1990;36:1760-4.