Directions for using Normalized Method
Decision Chart to Judge Performance
- Define the analytical quality requirement
for the test in the form of an allowable total error, such as
given by the CLIA PT criteria for acceptable performance in a
proficiency testing event. For example, TEa is 10.0% for cholesterol.
- Evaluate the imprecision and inaccuracy of
your method. From the replication experiment, calculate the SD
or CV for the method. From the comparison of methods experiment,
calculate the bias from t-test statistics if the mean of the
comparative method is close to the medical decision level of
interest, or use regession statistics to calculate the systematic
error at the medical decision levels of interest.
- Calculate your normalized operating point,
which is your observed imprecision and inaccuracy expressed as
a percentage of the allowable total error. For example, a 2%
observed CV would be 20% of a 10.0% TEa; a 1% observed bias would
be 10% of a 10.0% TEa. Use the Normalized OPSpecs Calculator
available below.
- Print the Normalized Method
Decision Chart.
- Manually plot your normalized operating point
using the coordinates obtained from the calculator.
- Judge performance on the basis of the location
of your operating point.
- A method with “poor performance”
does not meet your requirement for quality and is not acceptable
for routine operation.
- A method with “marginal performance”
will require a Total QC strategy that emphasizes well trained
operators, limited rotation of operators, more expensive statistical
QC (higher N, multirule type procedures), agressive preventive
maintenance, careful monitoring of patient test results, and
efforts to improve method performance.
- A method with “good performance”
can be well-managed in routine service if you plan the statistical
QC procedure carefully and are willing to spend the resources
necessary to implement a multirule procedure with 4-6 control
measurements per run.
- A method with “excellent performance”
is easy to manage in routine service even with widespread rotation
of many operators and can be controlled with minimum expense,
usually with a single-rule QC procedure and a minimum of 2 control
measurements per run.
|
Directions for Using Normalized OPSpecs Charts to Select
a QC Procedure
- Define the analytical quality requirement
for the test in the form of an allowable total error, such as
given by the CLIA PT criteria for acceptable performance in a
proficiency testing event. For example, TEa is 10.0%
for cholesterol.
- Evaluate the imprecision and inaccuracy of
your method. To define your operating point, express your observed
imprecision and inaccuracy as a percent of the allowable total
error. For example, a 2% observed CV would be 20% of a 10.0%
TEa; a 1% observed bias would be 10% of a 10.0% TEa.
You can use the Normalized OPSpecs Calculator available on this
website.
- Obtain Normalized OPSpecs charts, such as
those available here, for common control rules with N's of 2
and 4.
- Plot your normalized operating point on the
Normalized OPSpecs Chart for 90% AQA for
control rules with N=2. For example, the cholesterol normalized
operating point would be 20% on the x-axis and 10% on the y-axis.
- a. Inspect the 90% AQA chart and select a
control rule(s) whose operating limits are above your normalized
operating point. Identify the control rule(s) from the key at
the right side of the chart.
- b. If no QC procedure can be selected in
step 5a, try the 90% AQA & N=4 chart.
- c. If no QC procedure can be selected in
step 5b, and if your analytical method is very stable (and only
occasionally has a problem), plot your normalized operating point
on the Normalized OPSpecs Charts for 50%
AQA & N=2, and if that doesn't work, plot your normalized
operating point on the 50% AQA & N=4
chart..
- If no QC procedure can be selected in steps
5a-c, use a "maximum" QC procedure such as the multirule
procedure with N=4 or use the 2 SD limits with N=2.
- Select at Total QC strategy on the basis
of the error detection expected, as described in the Lesson
on Total QC Strategies on this website.
- Reassess for changes in method performance.
- To Recap:
First: Normalized OPSpecs Chart for
90% AQA, N=2
Second: Normalized OPSpecs Chart for
90% AQA, N=4
Third: Normalized OPSpecs Chart for
50% AQA, N=2
Fourth: Normalized OPSpecs Chart for
50% AQA, N=4
|