Trends

Re-engineering, out-sourcing, down-sizing, outcomes focus -- all the management fads have come to the healthcare laboratory with a vengeance. Dr. Westgard (with a little help from Dilbert) sorts out which approaches are valid and which are dangerous.

• Trendy Management Practices
• QC isn't trendy!
• QA and TQM are no longer trendy either!
• Outcomes are in and incomes are out!
• What are the difficulties in measuring outcomes?
• Need to consider many variables in the process
• Need to standardize medical processes
• Need for careful study design and data analysis
• Need for reliable data
• Difficulty of generalizing the findings
• Lack of stability of current processes
• What to do?
• References

Trendy Management Practices

Healthcare organizations have caught the bug for new management trends. It's probably the inevitable result of healthcare becoming a business and adopting modern business techniques - marketing and advertising, quality assurance, Total Quality Management, re-engineering, out-sourcing, etc. While there is much to learn from the practices and experiences in other organizations, there is also a danger in becoming like them.

Workers commonly complain about trendy management practices, as illustrated in the popular Dilbert cartoon strip. Most of us who read Dilbert suspect that someone in our organization writes under the pseudonym of Scott Adams. Many of our own complaints seem to show up in these cartoons. A common thread through many of these cartoons is that management seldom makes any serious comittment to solving problems, regardless of the trendy management practice being implemented. Just learning to talk a new management lingo seems to suffice, rather than doing any serious work to solve the problems that affect an organization's performance. The short-term appearance of change seems to be enough for managers to move up and to move on. There is seldom any long-term improvement because there is no constancy of purpose (remember Demings first point for quality management).

I would describe the current approach to laboratory quality management as a "fruitbowl." No, that's not another bowl game for football, but it does represent apparently competing QM approaches of which today's popular choices seem to be utilization control and outcomes assessments. It's a fruitbowl because these choices are what looks good and appeals to our tastes today, without recognizing our need for a balanced diet that integrates many different practices into a quality management system.

QC isn't trendy!

Quality control is an old established management practice in healthcare laboratories, dating back to the 1960s. QC is not in vogue today, particularly in settings such as Point Of Care testing where it might be particularly valuable. It's become unpopular because it costs money to run controls and requires some analytical skills and statistical understanding that may be limited in non-laboratory settings. Thus, there are a variety of efforts to eliminate QC, even though QC may still be the most efficient mechanism for monitoring a wide variety of factors and variables that affect the quality of a testing process. Arguments are being made that pre-analytical and post-analytical errors are the biggest concerns in these settings, therefore traditional QC isn't important or useful anymore. If QC can be eliminated in these settings, then it follows that QC probably should also be eliminated in large laboratories where there are even better analytical systems and more highly trained analysts. That will be the trend!

QA and TQM are no longer trendy either!

Quality assurance is a more recent management practice that began in USA laboratories in the mid 80s, primarily because of new accreditation guidelines from the Joint Commission for Healthcare Organizations (JCAHO). It is interesting to recall that QA initially focused on test utilization by calling on the laboratory to police the ordering of tests. At this same time, Total Quality Management (TQM) was emerging in USA industry and was focused on satisfying customer requirements. Healthcare QA was clearly out-of-step with the industrial model of TQM, and changes in accreditation and regulatory guidelines during the latter 80s led to widespread implementation of TQM in the 90s. TQM has since gone through it's popularity phase, and in many organizations is now being superceeded by programs in business process improvement, re- engineering, utilization control, and outcome assessments.

Outcomes are in and incomes are out!

These new programs are mainly driven by the need to reduce costs, while maintaining some appearance that quality is still important. With capitated and managed care, laboratories are no longer income generators or revenue centers and now have become cost centers. The real management strategy is cost-control, not quality control, and that's the main driving force for new programs in utilization control and outcomes assessments. Management may state that "Quality is a given" and imply that the necessary quality is already being achieved, assured, and controlled. Unfortunately, analytical quality is mainly being assumed, rather than being assured or guaranteed by current quality management practices [1].

[See Assuring Quality through Total Quality Managment for a discussion of the technical management the testing processes in laboratories today. My conclusion was that there's still a lot of work that needs to be done to properly manage the analytical quality of laboratory testing processes and that there's a real danger that this important work won't be accomplished if laboratories simply move on and adopt trendy management practices.]

What are the difficulties in measuring outcomes?

I'm not against measuring outcomes and there actually are some simple ways to do it. Looking at the big picture, we can simply count the people walking in and keep track of how they leave - on their own or carried out in a box. I don't mean to be crude, but measures of outcomes will likely be crude and difficult to understand without adequate information about the patients and the processes. The outcome for each patient is the result of a very complex healthcare process. There is little established methodology on what to measure and how to perform the studies, although there are some good research programs being started that will eventually lead to valid methodology. At the present, however, there are many problems that need to be overcome.

Need to consider many variables in the process

Patient outcomes are the result of a process that is a lot bigger than just the laboratory testing process. It involves the whole healthcare delivery process. The outcomes for a group are likely to depend on a variety of healthcare processes. Each of these processes involves a number of different departmental services, a number of different professionals, and a large number of variables in the care and treatment of an individual patient.The laboratory is a small part of that process, is at least one step removed from user of its product, and two steps removed from the consumer of the product. This distance is a problem because the outcome of the process will depend primarily on what someone else does with a laboratory test result. If they don't use it properly, then the outcomes will reflect that variable, not whether the test might have been useful. Furthermore, if the delivery of the appropriate treatment was flawed or the patient didn't follow the prescribed treatment, then the outcomes will reflect those variables rather than the usefulness of a laboratory test for establishing a diagnosis and monitoring treatment.

Need to standardize medical processes

The practice of medicine is often described as an art, not a science. Different physicians practice in different ways because of their education, training, experience, and preferences. Imagine a laboratory that operated in a similar way, allowing each individual analyst to decide on how to best perform a test, what standards to use, how to calibrate, what controls to use, what reagents, what conditions for analysis, etc., based on their individual education, training, experience, and preferences. Just as the development of standard measurement processes is the first step in managing the quality of laboratory tests, it is essential to develop standard medical practices, or critical or clinical pathways, as they're often called today. This is not an easy undertaking. See the Clinical Laboratory News article by Bowie [5, Implementing appropriate testing practices: The role of the laboratory in critical pathways], which concludes that a multidisciplinary team is needed and "the single most important determinant is the availability of a respected, knowledgable, and motivated physician leader."

Need for careful study design and data analysis

Given that there are many factors, variables, and people involved in a medical process, the study design and data analysis will be critical. Laboratory scientists are often more skilled in these areas than others healthcare workers, but it is important to recognize that even simple studies such as the method validation studies required by regulatory and accreditation guidelines and commonly performed in clinical laboratories are still fraught with problems of data analysis and interpretation.

For example, the correlation coefficent is still being used to justify the acceptability of method performance, even though this statistic is of little value and its use has been discouraged in the clinical chemistry literature for at least 20 years [6]. One journal is even considering outlawing the correlation coefficient and regression analysis [7], substituting a simple graphical description in the form of the Bland-Altman difference plot [8], and justifying this approach primarily on the popularity of the Bland-Altman paper as a "citation classic," even though there are many arguments about the validity of the difference plot [9]. The point is we don't always do a good job on the relatively simple method evaluation studies that are needed to evaluate and manage the outcomes of the new methods and instruments in our laboratories. In fact, many laboratories today depend on the manufacturer's technical personnel to perform the required method validation studies during installation of new systems; some labs even depend on those technical personnel to provide the laboratory with the documentation to show the next inspector. Does this experience suggest that laboratories are ready and able to handle the potentially much more complicated data analysis required for outcomes assessment?

Need for reliable data

These studies require accurate and reliable laboratory test data. That means the analytical quality of routine service methods today must be acceptable, which might be true when the methods are working properly, but certainly isn't true if there were any problems during the course of the study. If there were analytical problems, it is doubtful that medically important errors could be detected by current QC practices, therefore routine laboratory data may not be reliable. For example, in a recent assessment of the precision performance of USA laboratories [10], we found the necessary analytical performance was routinely available for only one test - potassium. For the other 17 tests we studied, the percent of laboratories having the necessary precision performance varied from 0% for digoxin to 72-88% for hemoglobin. These results confirm an earlier recommendation by Ross and Lawson that improvements in precision are still needed in many laboratories today [11].

Difficulty of generalizing the findings

Early outcomes studies may not be reproducible or applicable in other settings. The lack of rigorous methodology may lead to an exporatory investigation of the data. With enough data and easy access to a variety of statistical tests with PC software, something interesting can and will be found. However, critical variables between sites may not have been identified or standardized, thus the study may be unique and apply only to the individual situation. That's still okay if the study is valid and useful for that site, but there's always a danger of generalizing the findings instead of developing a general study protocol that can be applied at different sites.

Lack of stability of current processes

It does little good to measure the performance of an unstable process. If performance turns out to be okay, there is no guarantee that it remains okay because something will have changed. If there are problems, it's very difficult to identify them and fix them because some other things will have changed. Given the rudimentary standardization of the medical treatment process at this point in time (i.e., the early state of development and evaluation of clinical pathways), it is important to pick the process or subprocess carefully to minimize the number of other variables that may affect the outcome and to be sure that part of the process will remain stable during the measurement period. It's hard to image finding a stable process in a healthcare organization today, yet we seem committed to measuring the outcomes of these unstable processes anyway. That's okay as long as we recognize the results are of little value in improving the process. But, if we can't improve the process, why are we measuring its outcome?

What to do?

Quality management practices are expected to evolve. They usually start with inspection, then advance to statistical Quality Control and broader measures of Quality Assessment, which leads to methodology for Quality Improvement which in turn leads to Quality Planning approaches that are guided by defined quality goals and requirements. This quality management process must be implemented to provide optimal, stable processes that achieve the desired quality in routine service. Although this quality management process is reasonably well developed in laboratories, there usually needs to be more work on quality planning and quality requirements. This will lead to revised methods and QC procedures that are appropriate and cost-effective for each test, based on the quality required for that test, the precision and accuracy observed for the method, and the error detection and false rejection characteristics of the QC procedure.

This same kind of quality management process needs to be developed and applied in other departments and across departments in a healthcare organization. Efforts to develop clinical pathways are a beginning step for defining and standardizing quality medical processes, which in turn must be monitored and controlled, improved, and replanned until the necessary quality can be achieved. Utilization control should occur as a direct result of standardizing medical processes. Outcome measurements will be important for monitoring and improving these medical processes. With careful use and interpretation of individual tests in a specific medical process, the clinical quality that is required will be better defined and laboratories will be able to better optimize their testing processes and assure the necessary quality is achieved in routine operation.

Utilization control and outcomes assessments, therefore, are expected in the natural evolution of a quality management process and are essential for the development of well- defined and standardized medical processes. These new practices are part of an overall quality management system that is built on a solid foundation of analytical quality management (as shown in the accompanying diagram). This is an integrated and structured set of QM practices rather than the unorganized collection of practices represented by the earlier fruitbowl. Outcomes and utilization have a distinct place in this structure, as do other practices in analytical quality management.

Today it is still important for laboratories to continue to develop the management practices that are critical in providing a solid baseline for utilization and outcome studies, which means that laboratories must guarantee reliable, accurate test results during routine service. While we are accomplishing this, we should also participate in planning and standardizing medical processes to make sure that the right tests are utilized and that there are proper guidelines for test interpretation. We must do our part in the laboratory because no one else is capable of doing it. We also need to help others understand how to better manage their parts of the process. We are an important player on the healthcare management team because of the quality management practices and skills we have already developed.

References

1. Westgard JO, QA: Are laboratories assuring, assessing, or assuming the quality of clinical testing today? Proceedings of the CDC 1995 Institute on Critical Issues in Health Laboratory Practice: Frontiers in Laboratory Practice Research. CDC, Atlanta, GA, 1996, pp 179-189.
2. Westgard JO, Barry PL. Total Quality Control: Evolution of quality systems in health care laboratories. Laboratory Medicine 1989;20:377-84.
3. Westgard JO, Burnett RW, Bowers GN. Quality management science in clinical chemistry: A dynamic framework for continuous improvement. Clin Chem 1990;36:1712-16.
4. Westgard JO. Error budgets for quality management: Practical tools for planning and assuring the analytical quality of laboratory testing processes. Clin Lab Manag Review 1996;10:377-403.
5. Bowie LJ. Implementing appropriate testing practices: The role of the laboratory in critical pathways. Clin Lab News 1997 (January issue).
6. Westgard JO, Hunt MR. Use and interpretation of common statistical tests in method comparison studies. Clin Chem 1973;19:49-57.
7. Hollis S. Analysis of method comparison studies [guest editorial]. Ann Clin Biochem 1966;33:1-4.
8. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; i:307-10.
9. Stockl D. Beyond the myths of difference plots [letter]. Ann Clin Biochem 1996;33:755-7.
10. Westgard JO, Bawa N, Ross JW, Lawson NS. Laboratory precision performance: State of the art versus operating specifications that assure the analytical quality required by proficiency testing criteria. Arch Pathol Lab Med 1996;120:621-625.
11. Ross JW, Lawson NS. Analytical goals, concentration relationships, and the state of the art for clinical laboratory precision. Arch Pathol Lab Med 1995;119:495-513.