Basic Planning for Quality

You've read about this new Quality Planning process. But you're skeptical. Want to see proof? How about some real-world examples, showing bench-level quality planning in seconds. Well, this lesson provides an overview of needs, approach, and methodology for quality planning in healthcare laboratories. This overview should be useful as a preview, review, or quick refresher of the quality-planning process and the training materials available to support your applications.

• Quality planning overview
  • The WHY of quality planning
  • The WHAT of quality planning
  • The HOW of quality planning
  • The WHEN of quality planning
  • The WHO of quality planning
• Example applications
• References

This lesson provides an overview of needs, approach, and methodology for quality planning in healthcare laboratories. This overview should be useful as a preview, review, or quick refresher of the quality-planning process and the training materials available to support your applications. Because we all learn best from tests and methods that are of interest to ourselves, the remaining lessons present a variety of example applications for routine chemistry, toxicology, hematology, endocrinology, and immunology. Use these applications to master the steps of the quality-planning process and become proficient in performing the process. Proficient means you should be able to select a QC procedure and identify the TQC strategy in one minute or less. Even the one-minute manager can perform quality planning with support of the tools and technology available today!

Quality planning overview

A series of fifteen lessons are available to support your understanding and application of quality planning for laboratory tests. A manual quality-planning process is available as part of these training materials and can be readily implemented in any healthcare laboratory. Mini-courses, short courses, and workshops may utilize only a subset of these lessons, so you may find it useful to know about all the lessons that are available.

The first three lessons in this series [1-3] focus on the WHY or need for quality planning, the next two [4-5] describe WHAT should be done to plan the quality of laboratory tests, and the following three [6-8] provide the HOW or practical methodology for performing quality planning quickly and easily. This lesson summarizes important points about the quality-planning process and provides additional discussion about WHEN to do quality planning and WHO should be involved and responsible. The remaining lessons provide example applications for different areas of the laboratory.

The WHY of quality planning

1. The laboratory must provide a final and independent assurance of quality for the test results it produces. This need for an independent quality management process is described in A Wake-up Call for Laboratory Quality Management [1], which discusses the limitations of current laboratory management practices that bundle many of the essential quality management activities into the services provided by manufacturers of analytical systems. A current FDA action against a major manufacturer exposes several fallacies in current management thinking:

• Analytical quality is not a given!
  
• Responsibility for quality can't be out-sourced!
• Compliance is not enough!
  
• Statistical QC can't be eliminated!
  
• Minimum personnel standards are skills are not sufficient!

2. The principles of Total Quality Management (TQM) point out the importance of defining quality standards and establishing a quality planning process that builds the desired quality into daily production processes. Assuring Quality through Total Quality Management [2] describes a quality management framework that is constructed from components - Quality Laboratory Processes, Quality Control, Quality Assessment, Quality Improvement, and Quality Planning - and assembled in a loop or cycle that provides continuous improvement of quality (as shown in the accompanying figure). This quality management framework should be centered on, or guided by, quality goals or standards. In most laboratories, implementation places a priority on the following:

• Defining quality goals or standards
  
• Establishing a quality planning process

3. Current regulatory and professional practice guidelines identify the need for quality planning and provide guidance for implementing a quality-planning process. In Complying with Regulations, Standards, and Practice Guidelines [3], the TQM framework is compared with JCAHO IOP guidelines (Joint Commision for Accreditation of Healthcare Organizations, Improving Organizational Performance) CLIA regulatory rules (Clinical Laboratory Improvement Amendments), and NCCLS QC practice guidelines (National Committee for Clinical Laboratory Standards), which reveals that:

• JCAHO IOP strongly recommends quality assessment, quality improvement, and quality planning.
  
• CLIA rules emphasize quality standards, quality laboratory processes, quality control, and quality assessment.
  
• JCAHO and CLIA together support the complete framework identified earlier from principles of TQM.
  
• NCCLS provides guidelines for planning QC procedures as shown in the accompanying figure. The planning process includes steps for defining the quality required for a test, determining method performance, identifying candidate statistical QC strategies, predicting QC performance, setting goals for QC performance, and finally, selecting an appropriate QC procedure.

The WHAT of quality planning

4. A detailed quality-planning process can be developed by building on the NCCLS QC planning guidelines and introducing a quality-planning tool that provides information about the performance of different QC procedures. Devising a Practical Process [4] describes the steps of a quality-planning process that utilizing a graphical quality-planning tool - the chart of operating specifications, or OPSpecs chart. As shown in the accompanying figure, the two major areas of application are:

• Setting specifications for imprecision and bias when selecting a new analytical method, and
  
• Selecting control rules and numbers of control measurements when selecting a new QC procedure.

5. The first and most important step in the planning process is to define the quality required for the test. Unfortunately, this is not a trivial pursuit! Defining Quality Requirements [5] describes the difficulties and the solution:

• Concepts and terms are not consistent in the current literature on quality requirements, therefore several different formats exist, such as the medically important change in test results, the allowable total error, the maximum allowable standard deviation, and the maximum allowable bias.
• Multiple factors affect test variability and different factors are included in different concepts or formats of quality requirements.
• A system of quality standards must be recognized to understand the relationship between the different types and formats of quality requirements. The accompanying figure illustrates a system involving clinical outcome criteria, analytical outcome criteria, and method performance criteria.
• All current quality standards assume that method performance is stable and do not consider the need for QC to detect unstable performance.
• Quality standards must be converted into specifications for the imprecision and inaccuracy that are allowable and the control rules and number of control measurements that are necessary to detect medically important errors - the bottom line in a system of quality standards.

The HOW of quality planning

6. A practical tool for translating quality requirements into operating specifications is needed to make quality planning quick and easy to perform. Adopting the OPSpecs Chart as Your Planning Tool [6] provides the basic details about a graphical tool called the chart of operating specifications (or OPSpecs chart), including:

• How to read an OPSpecs chart and recognize the imprecision and inaccuracy that are allowable for different QC procedures.
  
• How to determine method performance specifications from the x-intercept of the operating limits of different QC procedures
  
• How to select a QC procedure by plotting the operating point of your method (observed bias as y, observed imprecision as x).
  
• How to assess the need for quality improvement by identifying changes in imprecision and inaccuracy that would lead to better and easier quality control.

7. Cost-effective operation of testing processes depends on individualizing the QC procedures for each test and method in the laboratory. QC is a technical management strategy and statistical QC is one component of the broad or Total QC strategy. Formulating a Total Quality Control Strategy [7] describes how to establish the appropriate balance between statistical and non-statistical components (preventive maintenance, instrument function checks, performance validation tests, patient data QC, in-service training) on the basis of the error detection capability of the statistical QC procedure for the application of interest. Three general strategies are pictured here:

• A HI-Ped strategy is appropriate when statistical QC provides 90% detection of medically important errors. Costs are minimized by selecting QC procedures that have a low number of control measurements and a low probability or chance of false rejections.
  
• A MOD-Ped strategy is appropriate when statistical QC provides at least 50% detection of medically important errors. Costs increase to achieve the maximum detection possible with the maximum amount of statistical QC that is practical. Preventive efforts and non-statistical QC increase.
  
• A LO-Ped strategy is needed when statistical QC provides less than 50% detection of medically important errors. The emphasis must be on prevention of errors through operator training, preventive maintenance, verification of individual performance factors through instrument function checks and method validation tests, and monitoring patient test results by correlation with other diagnostic information or by patient data QC procedures. Cost is high and efforts should be made to improve analytical performance.

8. Normalized OPSpecs charts provide a practical tool for doing quality planning by hand. Implementing a Manual Process using Normalized OPSpecs Charts [8] describes how OPSpecs charts can be scaled to be used with any quality requirement. As shown here, the y-axis is scaled from 0 to 100 and the x-axis is scaled from 0 to 50. The operating point is then calculated as a percent of the quality requirement and plotted on the normalized chart. By using this approach, only six charts are needed to select a QC procedure for any test. Everything you need to get started is available from the Internet:

• PDF file of normalized OPSecs charts can be downloaded from the Internet and stored on your computer. Print a new set of normalized charts whenever you need to use them.
  
• PDF file of a Worksheet for Selecting a QC Procedure can also be downloaded and stored on your computer. It provides a convenient way of documenting your applications.
  
• A Normalized Operating Point calculator is also available on the Internet to calculate the operating point that needs to be plotted on normalized OPSpecs charts.

9. The steps of the manual planning process using normalized OPSpecs charts are summarized in the accompanying diagram.

• Define the quality requirement at a medically important decision using the format of an allowable total error, such as given by the CLIA criteria for acceptable performance in a proficiency testing survey.
  
• Assess method inaccuracy as the %bias and method imprecision as the %CV at the medically important decision level.
  
• Calculate the y-coordinate [(%bias/%TEa)*100] and x-coordinate [(%CV)/%TEa)*100] for the normalized operating point.
  
• Plot the normalized operating point on the OPSpecs charts selected for the number of control materials to be analyzed (2 materials or 3 materials).
  
• Inspect the normalized OPSpecs charts in the order low N 90% AQA, high N 90% AQA, and high N 50% AQA.
  
• Select the control rules and total number of control measurements from the first chart that provides a solution (i.e., provides a QC procedure whose operating limits are above the operating point). Try to achieve 90% error detection with 5% or less false rejections. If no selection is found, use your maximum QC procedure.
  
• Adopt a Total QC strategy to balance the efforts expended on statistical QC and other non-statistical QC components. Adopt a Hi-Ped TQC strategy when the QC selection is made from a 90% OPSpecs chart; adopt a Mod-Ped TQC strategy when the QC selection is made from a 50% OPSpecs chart; adopt a Lo-Ped TQC strategy when a maximum QC procedure must be used.

The WHEN of quality planning

In a healthcare laboratory, quality planning should be an ongoing activity. It's never too late or too early to apply the quality planning process. You need to begin right away! Given that the quality-planning process can be fast and easy to perform - one minute or less, the biggest demand on your time will be the time needed to define the quality requirement for the test and the time needed to determine the imprecision and inaccuracy of your methods. Once you become proficient at using the quality-planning process, you can apply the process to accomplish the following:

• establish method specifications for purchasing new analytical systems;
  select QC procedures once you have determined the imprecision and inaccuracy from your initial method validation studies;
  
• adapt the QC procedures (if necessary) to accommodate the performance of your methods under routine conditions, as characterized by estimates of imprecision from routine QC data and estimates of inaccuracy or bias from monthly peer-comparison results and periodic proficiency testing surveys;
  
• identify tests and methods that need improved analytical performance;
  
• prioritize purchases of new methods, instruments, and analytical systems on the basis of needed improvements in analytical quality;
  
• assign experienced analysts to analytical systems that require technical expertise to assure quality;
  
• determine which methods and instruments are best suited for cross-training and widespread rotations of analysts;
  
• update the QC guidelines during the annual review of your procedure manuals.

The WHO of quality planning

In workshops and seminars, I often tell participants that I can teach fifth and six grade students to do quality planning and it would take less time than teaching adults. The point of that statement is to emphasize that we have a well-defined quality-planning process and it's easy to do if you follow the directions. If given a quality requirement, the OPSpecs charts for that requirement, and the figures for the method bias and CV, any fifth or sixth grader could plot the operating point and identify the line or lines above the point. The mechanics of using an OPSpecs chart are simple.

The things that are difficult in this process are (a) defining the quality requirement for the test and (b) determining the precision and accuracy of the method. Those steps require skilled and experienced laboratory scientists. In principle, the medical director of the laboratory has the responsibility for defining the quality requirements for the tests. In practice, that activity may be delegated to managers, supervisors, lead analysts, or quality specialists. In cases where CLIA has defined an allowable total error, that requirement can be used as the starting point. Anyone familiar with the CLIA regulations could look up the requirements.

Reliable estimates of method imprecision and inaccuracy depend on proper statistical analysis of sets of experimental data, initially from method validation studies and later on from routine QC, monthly peer-review comparisons, and periodic proficiency testing surveys. Here's where training and experience with method validation studies and routine QC are important. Some statistical skills are needed - at least the knowledge of which experiments and statistics provide reliable estimates of different types of analytical errors. In many laboratories, these skills reside with QA or QC specialists, R&D technologists, senior analysts, or experienced managers and supervisors.

Example applications for practice

Cholesterol is my favorite test! I use cholesterol as an example over and over and over because it illustrates many of the problems and difficulties you can expect to encounter with other tests. As a clinical chemist, I've followed the evolution of cholesterol methodology and quality standards for over thirty years. In many ways, cholesterol provides a model system that illustrates the importance of analytical quality management in making test results commutable, i.e., interchangable from method to method and lab to lab [9].

Commutability should be our goal for all laboratory tests, not just cholesterol. This will require that each laboratory carefully plan it's testing processes to assure the quality required by national standards and guidelines. The cholesterol story shows the importance of having methods that are highly specific, excellent analytical performance (imprecision and inaccuracy), traceability of standards and calibrators, comparability between field methods and a national reference method, and statistical QC for assuring the quality of routine test results.

Few tests and methods are as well characterized and well studied as cholesterol. Therefore, we can expect there will be difficulties in assuring the quality of many of the other tests being performed in our laboratories. Even so, we will gain many useful insights into our quality management problems if we apply the quality-planning process to all our tests. To help get started with tests you're familiar with, the remaining lessons provide example applications in the following areas:

• Routine chemistry (see QP 10)
• Blood Gas (see QP 11)
• Immunology (see QP 12)
• Coagulation (see QP 13)

REFERENCES

1. QP 1: A wake-up call for quality management.
2. QP 2: Assuring quality through Total Quality Management.
3. QP 3: Complying with regulations, standards, and practice guidelines.
4. QP 4: Devising a practical process.
5. QP 5: Defining quality requirements.
6. QP 6: Adopting the OPSpecs Chart as your planning tool.
7. QP 7: Formulating a total quality control strategy.
8. QP 8: Implementing a manual process using normalized OPSpecs charts.
9. Wiebe DA, Westgard JO. Cholesterol - a model system to relate medical needs with analytical performance. Clin Chem 1993;39:1504-1513.