Tools, Technologies and Training for Healthcare Laboratories

Quality on the Sigma Scale at Aga Khan Laboratory

A recent study from the Scandinavian Journal of Clinical and Laboratory Investigation details the quality indicator performance of the Aga Khan University Hospital in Karachi, Pakistan. Quality Indicators were measured across the total testing process: pre-analytical, analytical, and post-analytical. Can you guess where the biggest problems occurred? The results might not be what you expect...

Aga Khan University Hospital: Sigma-metrics across the Total Testing Spectrum

July 2015
Sten Westgard, MS

[Note: This QC application is an extension of the lesson From Method Validation to Six Sigma: Translating Method Performance Claims into Sigma Metrics. This article assumes that you have read that lesson first, and that you are also familiar with the concepts of QC Design, Method Validation, and Six Sigma. If you aren't, follow the link provided.]

This analysis looks at performance on the short-term Sigma-metric scale on three phases of the total testing process, the pre-analytical, analytical and post-analytical:

Error identification in a high-volume clinical chemistry laboratory: five year experienceError identification in a high-volume clinical chemistry laboratory: five year experience, Jafri L, Khan AH, Ghani F, Shakeel S, Raheem A, Siddiqui I, Scandinavian Journal of Clinical & Laboratory Investigation, 2015 July;75(4):292-300

The Aga Khan University Hospital in Karachi, Pakistan, has achieved the Joint Commission International Accreditation (JCIA). Thus, while some laboratorians may be tempted to dismiss any results from a laboratory in this part of the world, the Joint Commission considers this hospital as one that achieves the highest international quality standards in healthcare. This hospital is also the first in Pakistan to achieve ISO 9001:2008 certification.

Their study collected five years of quality indicator data from 1 January 2008 to 31 December 2012. The five year period included 6,792.020 specimens, which were tested to provide more than 3.5 million tests each year, for a total of 17,631,834 analyses.

Here's the benchmarking of their quality indicators for that time period:

Aga Khan Quality Indicators

The two most error-prone processes were not found in the pre-analytical phase, but in the analytical and post-analytical phase.

The most error-prone process was proficiency testing failure. 5.4% of the time, Aga Khan was failing their PT, only 3.1 Sigma. This represents a defect rate that is 5,000 times greater than the defect rates for improper collection, incorrect labelling, broken containers, lost specimens, etc. Considering that just a few PT events are spread out over the course of a year, a single PT failure may in fact indicate that tens of thousands of patient results were significantly in error.

Another interesting detail: "The average PT failure in five years was 5% (3.1 on the Sigma scale) but during these years none of the PT cycles showed PT failure (failure to attain the minimum satisfactory score for
an analyte, test, subspecialty, or specialty for a testing event)." Essentially, this means the bar on passing the PT is actually lower than 3 Sigma. Given that many PT programs allow 80% as an acceptable pass rate for surveys, this should come as no surprise. This points to a weakness in EQA and PT; you can stay in compliance, pass the surveys, and still be operating at an unacceptable defect rate. EQA and PT programs need to consider improving their standards (perhaps increasing the frequency will allow them to demand a higher pass rate)

The second-most error-prone process was a delay in the Stat testing. This is more expected. Indeed, I have a hard time finding a lab where the clinicians aren't unhappy with turn-around time and the speed of testing. Those metrics are continually evolving downward, with clinicians demanding ever faster results. A defect in TAT and time delays is thus a moving target; what used to be a fantastic turn-around time a few years ago is now considered too slow. Frankly, this indicator is always going to be a tough challenge for even the most sophisticated laboratory.

For decades, laboratories have been told that the real problem is in the pre-analytical phase. Aga Khan is revealing that it's actually possible to achieve Six Sigma performance in the pre-analytical phase, and that the greater problems lie elsewhere. We can all learn something important from this study.