ISO
MU Survey 2024: 200 comments
The 2024 survey captured hundreds of comments from respondents, sharing their unvarnished views of measurement uncertainty.
MU Survey 2024: The respondents speak
October 2024
Sten Westgard, MS
See all the Global MU Survey Results:
- The Global Practice of Measurement Uncertainty (minus the US)
- Europe sets the Standard
- The US stands alone
- Rising Asia, rising uncertainty
- Middle East, middling uncertainty
- Africa, where mu is new
- South and Latin America, a special case
More than 200 respondents added comments about their use – or non-use – of measurement uncertainty in their lab. We’re happy to share a healthy portion of all the comments, sorted into a few relevant categories.
The largest group of respondents basically asked for more help, more webinars, more tools for measurement uncertainty (Want 2 learn). The second-largest group, 15%, of comments, were positive about measurement uncertainty. The third-largest group represents comments admitting that the labs really don't know anything about mu. The next group of responses were basically negative comments about mu, 12.1%. Another 9.2% noted that they calculated mu but found it a challenge (Challenge to mu). Then there were 8.2% of respondents who are mandated to mu, but do not find it worthwhile. Finally, the smallest percentage just entered thanks in the comments for the conducting of the survey (Thank you, 4.3%).
It would be exhausting to list all of these responses, but below follows a significant selection of them.
Hating on the Survey
"[W]hilst this may be a useful exercise, for a predetermined purpose, the questions have been asked in a leading manner, clearly designed to make a point. the lack of knowledge about a method does not negate its importance, scientific and clinical validity or relevance. i wonder why there is no reference to metrological traceability within these questions, when there is a clear focus on bias and issues related to inclusion in MU - which is a valid point. the results of this survey are not going to give a balanced representation of the application of MU. the implication is that it is enforced due to accreditation/regulatory requirements rather than recognising the body of literature, that is peer reviewed, that is developing the understanding and application of this method. as an independent metrological valid approach cartoons and belittling representations of MU are neither a valid scientific argument nor a constructive means of credible discussion. MU is recognised by ISO as an effective model for assay performance, directly linking it to clinical application. while this may be in opposition to other models, the discussion should be where each fits in, and a discussion of the benefits and pitfalls of all methods. i look forward to seeing how the results of this survey are presented."
The goal of the survey was to ascertain the practical laboratory implementation of measurement uncertainty outside the realm of plenaries, conferences, studies and standards. Westgard always has a strong perspective, I doubt that anyone in the audience of survey respondents thought we were that’s never something we pretend we don’t have.
We acknowledge that there is a massive institutional pressure to measure uncertainty. We Our aim with the survey was to probe how much that is actually happening in the real world. Our findings are not surprising: most of the laboratory world pays mere lip service to measurement uncertainty.
Certainly, our findings are open to refutation. Conduct another survey of more than 3,000 laboratories across more than 100 countries and see if you get different results.
There is a chilling implication in the tone of this criticism: that we do not have the right to conduct the survey, to ask questions, to tabulate results, and to present those results to the public. Further, that there is only one tone allowed for the discussion of measurement uncertainty, and that levity, word play, and sarcasm are inappropriate to the debate.
We admit to making puns, homonyms, and writing outside the generally accepted academic scientific style. We admit to asking pointed questions in our survey. If that makes our text easier to read and understand, and our results more revealing, we sincerely apologize.
Don’t know much about MU
“Most Medical Technologist in my country are unaware and not aware of the importance if Measurement of Uncertainty.”
“I am the Quality Manager for our lab, and I know very little about measurement uncertainty. No one has ever brought up the subject, so I don't know if it would be useful or not. To have an opinion, I will have to learn more about it.”
“I don't really understand MU so although we supposedly do use it, I don't know how it affects results in the long run.”
“We really don’t understand MU and it’s important[ce] in the lab”
Positive Proponents of MU
“We use MU since 2011, a[n]d [b]ecause of it we know the MAU for the future results (e[v]en before any bias oc[c]urred). [F]or example we had a MAU of 6% for glucose at 2011 and we still have it now. We also harmonised all of our chemistry anslysers of our 3 labs of our 3 hospitals. We also use MAU with patients data to ascertain that [o]ur patients resu[l]ts never surpasses[stet.] our MAU when MAU is lower than Intraindividual biological variation of the most fragile persons of our population. We also used TE along [w]ith MU between 2011-2015 and found that[ ]we could[ ](and should) leave TE.”
“UoM was calculated here as 2xCVA2 based on IQC measures. An UoM range is provided on monthly reports but the UoM has not been calculated for some time. We also state that UoM is done at our method verification stage.”
“I strongly recommend that lab results should be reported with MU wherever applicable. The change should initiate at the level of clinical practice guidelines like NICE in UK , NLA, ADA, AHA, ESC etc. If these guidelines recommends[stet.] MU data along with Sensitivity, Specificity, NPV and PPV for the diagnostic test, this will create awareness and a[n] understanding of MU concept to Clinical colleagues. Though there are multiple challenges in implementing MU, major challenge that needs immediate attention is harmonisation of testing methodologies between manufacturers. This is essential becoz[stet.] adopting different MU ranges for a single analyte by different manufacturers will lead to lot of confusion to clinicians when she / he is reviewing reports from two different labs. It is quiet[stet.] possible that clinician may insist to get a equipment or a method which has very low MU. The calculation or formula for calculation of MU also needs to be standardised and made simple to encourage widespread adoption.”
“The use of Mu is changing the lab where a lab person works. In the previous lab where I worked, we calculated and recorded the MUs but have never reported them in a lab report. Why we did not report any of it? Because clinicians did not know how to evaluate them. I believe still they do not know.”
“We do find the concept useful, but we also lack the knowledge required to understand how to properly apply the concept of MU to our processes. Also, just like Sigma Metrics, we have no issue finding theoretical information about MU, but applying it in our processes is very challenging.”
“I think it’s a great tool to estimate the uncertainty around your measurements based on the bias and impressions associated with it. It gives us an estimate of future bias estimate on EQA surveys and imprecision on the quantitative QC measure on the new lot.”
“In our organisation, we are required to calculate MU every 6 months. It gets reviewed by our Pathologists and management, however, there seems to be reluctance to abandon tests/assays that demonstrate high MU, despite several attempts to improve it eg increase frequency of QCs etc.
“Where there is also no specifications for esoteric tests, we include bias in the calculation by using the targets from the manufacturer's QC range.”
“Our lab only calculates UOM on an annual basis, although we would like to increase the frequency.”
Labs Challenged to Measurement Uncertainty
“The uncertainty consists of many steps, and this makes it challenging to calculate the total uncertainty. Therefore, we approach clinicians only with analytical variation.”
“Measurement uncertainty is [a] real challenge and it is not much in use because few people have better understanding on this, including me.”
“[S]till calculate it because of ISO 15189 said so, but haven[‘]t used for any clinical decision of the test measurement, and yet we will have difficulties to modify the LIS if the clinician ask for MU in the future in the reporting system.”
Negative views of Measurement Uncertainty
“Clinicians actually have never inquired about uncertainty.”
“We deal with it because it's an ISO 15189 requirement but its usefulness in practi[c]e is nonexistent. It's a hobby of fastidious nerds that got out of control if you ask me. Only RVC is a useful tool that has a practical application in clinical decision-making.”
“Oftentimes the conversations around MU get very complicated and the statistics nerds use it as a chance to flex their muscle - most of us find some of the calculations quite complex!”
“Despite the recent increase in uncertainty calculation in clinical laboratories, we saw that it is still an open discussion and not so much a practical and routine application of clinical laboratories in Portugal.”
“The idea that the calculation of uncertainty is a reality more for reference laboratories and that its calculation does not add added value to patients' quality control, are factors that delay its widespread application in Labs.”
“In routine laboratory practice, its application does not add value and does not allow identifying whether the error is predominantly due to lack of precision or accuracy.”
“As we use Bio-Rad's Unity, the calculation of uncertainty is automatic, but it is not a parameter that we analyze to evaluate the quality of our processes.”
‘If regulatory authorities require its calculation or the market considers it a requirement, we are ready to start applying it.”
Don't currently calculate Measurement Uncertainty
“We don’t use because there is not a consensus about measurement uncertainty with clinicians and in the government arranged protocols. “
“We don't use MU in my laboratory as we are yet to be exposed to it's[stet.] significance.”
“We already learned about uncertainty, it[‘]s more complicated a[n]d because the regulation dont ask about it so we havent [started] using it yet.”
Mandated to calculate Measurement Uncertainty, and not happy about it
“It's a requirement for ISO 15189, another thing to do and nobody asks for it except for the accreditation inspectors.”
“The concept of the use of measurement uncertainty only happens in accredited labs in our country to fulfill the ISO 15189 requirements. Competence for calculating it is still a challenge. Utilization of MU by the test requestors is also a big gap.”
“We only calculate MU because is mandatory in ISO 15189 accreditation. For performance evaluation we use other approaches: Six Sigma calculations, TEa.”
“The new ISO 15189 2022 wanted us to calculate MU for qualitative tests and we have to justify if we are not estimating MU for these tests. More non value added work!!!”
“UM is a requirement from Accreditation Canada Diagnostics based on ISO15189. It[']s never understood by Lab staff and physicians.”
“It is anticipated that our laboratory accreditation requirements will change to include MU calculation in the next year or so. Our accrediting body was waiting for the publication of ISO 15189:2022 to be published before making this change.”
“Our accreditation program PALC will insert a new requirement on MU for educational purposes only, in 2024. We are ISQua accredited and must harmonize our program to ISO15189.”
“In Sri Lanka, clinicians even do not see importance of calculating measurement uncertainty. Thus, the use of measurement uncertainty is only required when obtaining accreditation.”
“Used only to comply with ISO 15189 regulations.”