Interviews

Muhammad Sabry Saleh describes some of the unique challenges of laboratories running QC in Egypt, but finds many more challenges that are universal to labs worldwide. 

Challenges in Clinical Chemistry QC Practices

What the standards and guidelines documents DON’T tell you

July 2020
Muhammad Sabry Saleh

Why start from Clinical Chemistry?

Clinical Chemistry QC practices are among the most standardized with respect to other fields. There are many reasons for this, the availability and stability of QC material, well-established guidelines and the metrological standardization of the methods themselves, among them. Laboratory staff training typically starts with the standard chemistry QC process. We also advise lab professionals in other disciplines to attend and study the same course even if not fully applicable in their specialty, because we believe the lessons from chemistry QC can be extended to other laboratory testing areas (Hematology, Serology & PCR Molecular Biology).
Here, however, I will write some of the hidden truths on the challenges of our assumed “standard” QC practices.

Where are the standard requirements?

Quality is defined as the conformance to standards and requirements. In the quality community, we search for compliance with norms, standard documents and supportive guidelines. Through my career, I have met experts who feels irritated when there is no reference cited for a specific practice in the QC process.

But if we are brutally honest, in fact, the belief that QC Practices are “strictly standardized!” is a Myth!

When reviewing QC processes, we find many practices, of which some are essential in required QC/QA planning, are NOT standardized in international documents or consensus guidelines. For example, ISO 15189 gives a few, articulate, statements about QC planning such as requirements for proper ‘Design’ that “verify the attainment of the intended quality of results” and about QC frequency that is should be “based on the stability of the procedure and the risk of harm to the patient from an erroneous result”. Even CAP checklists, JCI and Canadian standards that are considered very comprehensive documents, do not fully flesh out the details on QC implementation, instead referring back to ISO & CLSI.

CLSI C24 stands as one of the most well-established guidelines for introducing a ‘fundamental’ roadmap.

The lack of comprehensive instructions on QC practices in most regulations and guidelines are built on an assumption that the significant variation in clinical laboratories worldwide (with regards to workflow, operations, assays, testing volume and national regulations), it’s unreasonable to provide a single guide or roadmap for laboratory professionals. That is, most of the regulations justify skimping on details by saying there is too much variation in the laboratory. It’s a convenient way to avoid taking responsibility.

What should the Roadmap should be like?

Apart from the detailed cause analysis for this lack of harmonization & guiding, we can always take the initiative to talk about our practices, share experiences and publish improvement approaches. Here I present highlights on formulating a roadmap that, in our approach, should be based on THREE PILLARS:

1. Assure the right concepts are in the laboratory foundation.
2. Construct an Integrated Plan
3. Link technical QC process with managerial and leadership requirements tied to significant impact.

1- Getting the Foundational Concepts Right

Laboratory QC processes usually considered a leading example for Healthcare. Our QC practices have imported industry and statistical concepts to enrich our insights on variation, error and patient safety. On our way for standardization and improvement, we should back review and ensure the reliability of the basic foundational concepts and practices.

QC Limits as a Central Practice

Recently, I have encountered a lab training lecture in Egypt that debated whether we should work on 2SD or 3SD limits! This was a strange discussion, since it didn’t address the most basic issue of what kind of QC limits/values are defined? Are they manufacturer QC limits or lab-established values? In Egypt, the majority of labs, (and here I mean those labs who are actually in compliance with country and international standards, not those who operate outside of compliance) are NOT familiar with the concept of establishing their own mean and standard deviation.

Quick example: Working with manufacturer ranges can lead to a compromised Cholesterol results around the risk decision limit:
From real data in our labs, QC manufacturer SD value for Cholesterol method is 19.5 mg/dL, while on our calculated data SD is only 4.18 mg/dL. This means working with 2-SD limits can render accepting, for example, a (50) mg/dL deviation! Here you are exceeding, and not even close to, the Cholesterol quality goal of 6 % (RCPA goals) or the wider CLIA 10 %. For clinical evaluation, think of the consequences of such deviation if around the universal decision value of 200 mg/dL.

Ensuring Validity of Basic Routine Calculations

Summation of SD/CV over different lots. In the standard statistics references, you can find the term “Pooled Standard Deviation for Two Samples” where SDs from samples of different number of points are averaged by weight and root of averaged squares (1). In routine lab imprecision estimates, if you calculate SDs across QC lots you should consider calculating the pooled SD not the whole set of data. Also, consider the calculations across different reagent lots if considerable mean shifts occur. You can do a simulation on an Excel sheet to see how neglecting this simple notice may result in over-estimates of cumulative CV. This impacts the laboratory when they establish SD for a new QC lot. Why do many manufacturers pay so little attention to such simple calculations? And instead put measures of all cumulative sets of data together?

Fine Adjustments of ongoing QC limits: (common sense with evidence-based practices) It’s common to use Historical SD/ CV % when establishing new QC SD, instead of using the 20-points-only traditional approach. However, I have frequently encountered situations where lab technologists are confused and troubled because the resulting ranges are tight or inappropriate. You should ensure fine adjustment of mean that may be more important if SD is very tight so that mean placement will make a difference!

It’s very common to encounter inappropriate ranges. In one lab, due to lack of fast review of newly established QC ranges, many false rejections occurred, associated with QC repetitions, re-calibrations and changing reagent bottles. Subsequent strict evaluation and monitoring of limits resulted in updates that resolved the issues and significantly reduced QC repeats.

Evaluate your CV % - an evidence-based approach:
If we are suffering absence of sufficient guidelines, we should at least shift to EVIDENCE-BASED practices. Here is some advice:

• Ensure you have reliable cumulative SD/CV.
• Evaluate your limits against: QC Manufacturer values – Method published CV – EQA reported data.

Example: We find the calculated SD of ALT on one analyzer is exceeding the SD established from cumulative data. There is no technical reason to replace the old CV with a wider one. Instead, we examined EQA performance to ensure we have no indicator of poor performance and compare the peer group interlaboratory CV%. We use this evidence to justify an increase of our limits. However, the case is recorded for careful follow-up to ensure there are no underlying cause for increasing imprecision. Note that it’s not recommended to directly establish your QC limits using EQA data, but it may be considered an evidence-based approach to use them as an aid in your integrated QC evaluation and monitoring! This is a fine but important difference!

Recognize Bias and lower P:fr !

Rules: 4:1s & 10:x are commonly adopted in many labs without careful study. They can be problematic if there isn’t meticulous follow-up for unusual events (e.g., reagent lot change). 10:x violations can be diminished with careful adjustments of mean. More importantly, both 10:x and 4:1s rules are not necessary for rejection when the analytical performance is good (higher Sigma metrics). Consider using CUSUM and/or EWMA for bias recognition (this is very difficult without supporting software).

2- Integration of approaches in QC/QA Planning:

“The successful standardization status achieved in some clinical chemistry assays like Glucose, Cholesterol & Glycated Hb was the result of integrated efforts & large-scale studies between scientific clinical institutes and Metrology including IVD manufacturers. It’s until this collaboration occurs again, when we can reach harmonization in many other required assays” Quoted from a discussion with Tamer O. Gad, a laboratory CEO & IVD expert in Egypt.

Consider a common problem in Healthcare, and in other aspects of life, where we suffer from fragmentation of effort and lack of communication. Seeking references in QC planning, you will meet these sources:

• CLSI C24: one of the best guidelines there, with the most foundational roadmap. I always teach its sequence in my training workshops and internal lab staff orientation. The last edition, A4, integrated what is called risk-based thinking with relation to patient safety. However, many practical approaches and required calculations are not clear or not stated at all within the document
• CLSI EP23: a famous guideline by CLSI that establish for another foundational concept: Laboratory QC based on Risk Management. Strangely, this guide has stood alone for many years without being updated, supplemented with required practical points, or linkage to other guidelines!
• IQCP: A great philosophy from US that could, theoretically, be utilized to engage QC planning with risk assessment and render acceptable individualized protocols. However, it was formulated in the context of US regulations and the POC industry. So, it needs many extensions and modifications to be a basis for a globally integrated guideline.
• Sigma Metrics: As Prof. J. Westgard used to do, importing SQC concepts from industry with elegant ideas for implementation in laboratory medicine was globally accepted. While not clearly adopted in many international guidelines, Westgards continue introduce and update Sigma metrics-based QC. Westgards gave appreciated efforts in here-named INTEGRATION philosophy. In some of their publications (2), you can find a roadmap for QC planning that put basic QC practices, Risk assessment, MU, and Sigma Metrics all in one picture.
• Other guides from IVD manufacturers or European MU-based guides are available but completely separate from any integrated plan.

Highlights on some selected Requirements of an Integrated-Roadmap

•  External Quality Assessment: In a recent webinar on Proficiency Testing, I discussed the “Enhancing the utility of PT participation”. This can’t be done if the laboratory takes the “We-Passed-We-Are-Ok-We-Aren’t-Going-to-Change-Anything” approach. EQA and PT was meant to empower labs to monitor bias, trends, and concentration-based defects, extracting useful information and integrating output with internal QC results.
• Risk Assessment: Let’s admit that Healthcare Quality specialists, in general, are more trained in quality approaches & tools than laboratory professionals. On the other hand, laboratory quality specialists dig deep into technical & operational concepts that healthcare specialists are unfamiliar with. Risk assessment is a manifestation of this situation. In Egypt, for example, laboratory professionals handle risk management as a documentation compliance issue if accredited to ISO 15189. This results in reverse-definition of the process steps and mere documenting of them in risk management records! It is rare to encounter a REAL systematic risk assessment such as those seen in industry or hospitals. I highly recommend that labs do a real risk assessment and extend it to analytical phase.

The assumption is that risk assessment, even in its immature implementation, can still lead to important results. In the US IQCP experience, CDC-CMS provided simple risk assessment templates (3). Though criticized for the absence of Quantitative calculations, they still introduce good way to explore/document/recall vital aspects and weakness points that may affect the intended quality. This should be done in parallel to the routine SQC procedure. Do not rely on one single tool for monitoring and control. Even traditional risk management can raise FOCUS and lead to taking proactive measures.

Think of the following examples:
- Assays sensitive to water quality.
- Reagent on-board stability & poor overall stability of reagents of some manufacturers.
- Temperature-dependent enzymes and status of analyzer thermos-pieces.
- Semi-disposable reaction cuvettes and their REAL-required replacement intervals.
- Accumulating imprecision & risk of unacceptable deviation in case of old machines.
- Assays critically sensitive to pre-analytical conditions.
- Clinical significance of the assays.

We know well that experienced technical lab staff understand all the previous points. However, the prevalence of the “Continuous Production” model and pressure to meet TAT can leave weak points insufficiently monitored and expose the process to serious errors. Systematic Risk Assessment is still potent: it can detect problematic assays with ease. It’s time-consuming but with effective leadership and communication policies, it can give real solutions. I hope that CLSI EP23 and other risk-based guidelines will be merged and harmonized for better results.

• Sigma Metrics: Based on the concept of process capability from industrial engineering: “Matching the capability of the process with the range of the specification/tolerance interval for a given quality characteristic” (4), Sigma metrics are described by Westgards and others in many papers and guides (5). Westgards continue to explain how SQC planning is driven by characterizing methods on the 6-Sigma scale. On the theoretical basis, capability indexes and sigma metrics offer promising, evidence-based approaches for benchmarking performance and managing analytical quality with care. Practically, significant challenges face labs trying to implement this approach. In Egypt, many labs are curious but few are implementing Six Sigma metrics. Below, I will summarize a simple real case from our laboratories to elucidate:
  
  CASE of Sigma metric calculated for: Albumin BCG method on Beckman Coulter AU 480 analyzer.

• • Sigma indicator observed/calculated at a decision value near 2.5 g/dL.
  • CV % calculated from the intermediate precision of last three months after removal of outliers.
  • Bias % averaged from Four EQA/PT results vs PT mean of comparison. Results were taken from two different EQA programs at results near the same observed concentration (Last Six months)
  • CLIA Albumin 10 % goal was used
  • Sigma calculated as (TEa% - Bias%) / CV%.
  • Results:  Alb BCG ~ 2.5 g/dL (10 - 3.5) / 1.86 = 3.5
  • This method is running in a lab where routinely there are no more than 50 samples run for ALB. More time and space are needed to present literature review, so far selected publications are used as references for proposed lab actions that could consider the following options and notes: (6) (7) (8)

1. 1. Implementing MR: N4 (i.e., 1:3S/2:2S/R:4S/4:1S with Four levels of QC per day)
   2. Running as a QC startup design, could be supplemented according to maximum run size and desired reporting. In this case, one level (N1) with 1:2S rule could be used as a bracket monitor after 50 samples.
   3. You can shift to 1:3S N4 instead but accordingly the power of error detection (Ped) will fall to (0.68). THIS SAME LATTER RULE WOULD YIELD Ped: 0.98 in case of 5.0 Sigma!! This how sigma metrics work!
   4. According to Westgards in 2019 (5), methods with Sigma (3.5-4.0) with run size > 40 samples can yield no more than 0.73 Ped (Maximum we get from 3.5 Sigma).
   5. As per Revupalli, et al, (8) comparison was discussed between Westgards and Cooper guidelines for rule selection and SQC strategy. (3.5) Sigma method is described as a “poor performer” with combination of rules required along with two levels of QC twice per day (Cooper guideline).

Even though the Sigma metrics philosophy could lead to more ease in QC strategies, practically it also introduces stringent QC strategies for those “poor performer” methods that are less than 4 Sigma. The above case is a method of a reputable analyzer/company in a lab where the PT results all passed and they experienced an overall stable QC. Running basic 4-6 QC levels in a small-sized lab that run < 50 samples for such test is problematic in laboratories where economic pressures translate into cost concerns. The whole picture is different if this was the case of a high-volume laboratory. Also, the above Sigma metric calculated for Albumin is not the final verdict. We are progressing to study and evaluate many running methods hoping to find reasonable/feasible solutions. In many cases, customization is NOT an option: variation in testing volume, equipment, work flow and other operational situations lead to different needs. This is part of integrated roadmap encouraged here.

“What cannot be completely attained, should not be completely left”, an old Arabic proverb

This should be kept in mind, while we are seeking and encouraging standardization and harmony in QC practices, this is not in conflict with individualized plans. A harmonized roadmap should guide you through this with EVIDENCE-based, integrated approaches. This is the goal!

Remember, too, Sigma metrics are risk indicators! Even if your laboratory is not implementing a specific combination of control rules, control measurements, and a selected QC frequency based on Sigma metrics, it’s highly recommended you pay more attention to your low-sigma methods. This will require more effort than what is mandated by your inspectors or accreditation surveyors!

3- Leadership Impact: Link technical QC process with managerial/leadership requirements of the most  significant impact.

Labs should evolve from an un-organized continuous production model to an organized model with well-defined processes. QC processes with diverse needs and actions require high collaboration/coordination level between quality and technical teams. Even though quality specialists, by definition, are facilitators with supportive functions, in the real-world, they sometimes become leaders or mentors of initiatives for quality improvement in cases where other technical teams are overburdened with ‘production’ pressures and other technical/clinical issues or they lack the latest knowledge of quality and guidelines. This should be handled with care.
Lab leadership should encourage improvement plans and quality initiatives. Technical teams need to share their vision and objectives and coordinate to implement them through policies and processes. Education and training should be valued. Quality improvement should be implemented through change management frameworks. In our labs, newly implemented QC procedures replacing wrong practices are powered through DMAIC projects. The very first part of establishing new ranges and related steps are implemented through simple PDCA.

Deeper look: The Revolutionary Role of Informatics!

In the era of information and communication revolution, data science and internet of things, we cannot only depend on traditional ways of establishing and implementing laboratory QC/QA. In fact, most of these discussed topics can be transformed through informatics solutions. In Egypt, we have lower level of informatics solutions implemented and LIMSs rarely linked with QC modules and data. As a researcher in statistics and data science, I know some skill in statistics is highly recommended for on-the-job training of laboratory professionals. This helps accomplish two goals:

• Establishing the foundational concepts right! I have met many lab staff who do not understand, for example, the philosophy behind SQC or do not know what “EWMA” is. How can they reliably select the best SQC procedures and measures if they lack basic understanding?
• Labs should push vendors for more scientific & evidence-based methodologies. Being good in statistics and data analysis adds to your competencies. Remember that IVD manufacturers and Lab Information System developers sometimes lack the expected robust knowledge in mentioned areas.

In addition to traditional statistics, automation and informatics solutions provide a greater capability for achieving complex and time-consuming actions required for the Integrated QC plan, When informatics power the solution, you can use customized algorithms, patient moving averages, real-time monitoring and more!

Informatics is no longer a ‘future’ vision. Rather it is the action plan for today!

Muhammad Sabry Saleh

References

1. Introductory Statistics, Prem S. Mann, 8th edition.
2. Six Sigma Quality Management System and Design of Risk-based Statistical Quality Control- James O. Westgard, Sten A. Westgard, CLINICS IN LABORATORY MEDICINE Volume 37, Number 1, March 2017.
3. CMS IQCP Link: http://www.cms.gov/Regulations-and Guidance/Legislation/CLIA/Individualized_Quality_Control_Plan_IQCP.html
4. Statistical Quality Control- M. Jeya Chandra, 2001 CRC Press LLC
5. James O Westgard, PhD, Sten A Westgard, MS, Establishing Evidence-Based Statistical Quality Control Practices, American Journal of Clinical Pathology, Volume 151, Issue 4, April 2019, Pages 364–370
6. Planning Risk-Based Statistical Quality Control Strategies: Graphical Tools to Support the New Clinical and Laboratory Standards Institute C24-Ed4, Hassan Bayat, Sten A. Westgard, and James O. Westgard, JALM, 2017.
7. Evaluating analytical quality in clinical biochemistry laboratory using Six Sigma, Mao X. et al., Biochemia Medica, 2018.
8. Revupalli, Suneetha & Ganji, Suresh Babu. (2019). Evaluation of Quality Assurance in a New Clinical Chemistry Laboratory by Six Sigma Metrics. Journal of Clinical and Diagnostic Research. 13. BC04-07. 10.7860/JCDR/2019/40658.12666.

About the Author

Muhammad S. Saleh is currently the manager of quality and consultation at Accu-Clinic diagnostic company where he operates the company compliance and consulting activities and manages ESfEQA technical support in Egypt (European Society of External Quality Assessment). He is also the director of quality & accreditation in one of private laboratory networks. M. S. Saleh holds a B.Sc in Chemistry and Biology, Cairo University.

Between 2011- 2018 with laboratory science background and experience of (5) years as a clinical lab chemist, he tracked to specializing in quality through NTC-Consultation, a reputed consultancy in laboratory medicine quality & accreditation, through which he worked with governmental, academic & private sector labs and acted as the deputy QM in one laboratory organization.

M. Saleh is dedicated to laboratory SQC,QA and EQA with PGD in Applied Statistics (Cairo University FGSSR). However, He is an inter-disciplinary professional. Currently he studies Computer and Data Science (CU FGSSR) and participates in the R&D of a newly LIMS as part of his passion & interests in laboratory management, quality improvement and informatics - He is Certified Health Informatician, Ministry of Health program.