Guest Essay

Jim Pierson-Perry and Oswald Sonntag discuss the German RiliBÄK rules. These new guidelines set new specifications for quality, both for performance of laboratories in EQA as well as routine operations. New concepts like the %Root Mean Square Deviation are introduced and explained.

October 2009

Jim Pierson-Perry
Senior Key Expert-Biochemistry
Siemens Healthcare Diagnostics

Oswald Sonntag
Senior Scientist
Ortho-Clinical Diagnostics

See an unofficial English translation of the RiliBÄK guidelines.

German law mandates a set of quality regulations, known as the RiliBÄK, which govern medical devices in laboratory medicine. These regulations are further tied to the European IVD directive and the ISO standards e. g. 15189. They are updated periodically to reflect changes in clinical need of regulated analytes, as well as improvements in technological capability of the devices. The most recent update was proposed in 2007. It is in partial effect now, with full implementation due by April 2010.

What are the RiliBÄK? Richtlinien der Bundesärztekammer

RiliBÄK  are literally the Guidelines ("Rili") of the German Federal Medical Council (BÄK). Much like CLIA limits in the US, they are meant to provide minimum requirements for the quality of quantitative test results in medical laboratories. Unlike CLIA, these guidelines do not cover any qualitative tests yet. An update with qualitative and also serological assays will follow within the next year.

The RiliBÄK quality requirements are synchronized with the well-known ISO 15189 principles and standards. But going beyond the ISO generalities, the RiliBÄK also set specific quality requirements / allowable errors / uncertainty recommendations. The 2007 guidelines mandate specifications for 67 analytes in serum and whole blood, as well as 10 analytes in urine, and 7 in cerebrospinal fluid. While that list may seem small, the guideline also give a method for determining specifications for any unlisted analytes, as well as the permission for laboratories to apply the specifications of listed analytes to similar unlisted analytes .

A laboratory must do proficiency testing (PT), external quality control, or peer comparison testing for any analytes that they test which is listed in the RiliBÄK tables. For those analytes not listed in the RiliBÄK, laboratories are strongly encouraged to do PT, but is it not mandatory. For the analytes where PT is mandatory, if the laboratory fails two successive PT events for the same analyte, a report must be filed with the government. Germany's equivalent (BfARM) of the FDA is to receive the notice and investigate the deficiencies. A fine up to 25,000 Euros is possible, although it is not yet known how these failures and investigations will be handled.

The RiliBÄK guidelines are somewhat unusual in that they don't describe total allowable errors, as we think of them in the US. Instead, in some cases the specifications are supposed to be written directly onto the daily control charts. Here's the difference: In the US, CLIA sets an allowable error limit for the result but labs still use 2s, 3s, etc. limits on their charts. In Germany, the laboratories are encouraged to draw the error specifications directly on their control charts.

Another unique aspect of the RiliBÄK is that the guideline cover both internal assessment of quality as well as external assessment. The internal assessment guideline are designed to assess a laboratory's quality using only a manufacturer-supplied target value and the laboratory's routine QC data. These assessments are done on a daily basis as well as retrospective over a period of time (referred to as a Control Cycle). The external assessment guideline is for comparing laboratory results with those of a peer group, proficiency testing group, or other external quality assessment.

How were the RiliBÄK recommendations developed?

The German Federal Medical Council conducted  several surveys to determine the specifications. At first a large number of laboratories was surveyed, then a smaller working group of 20 laboratories was surveyed again. This workgroup was selected to contain representatives of all types of laboratories, from small clinics to large reference labs. Preliminary quality targets were calculated from data submitted by the laboratory working group. After trimming 10% of results from the extremes, the remaining 90% range of values was used to formulate the final guideline

%RMSD = Root Mean Square Deviation

The 2007 RiliBÄK release introduced a new quality metric for internal assessments, Root Mean Square Deviation, abbreviated as %RSMD, and expressed as a percentage relative to a target value. Previous versions of the RiliBÄK used separate quality goals for bias and imprecision. These were replaced by the %RMSD in a move toward a total analytical error type of metric, expressed in a way similar to the ISO concepts of uncertainty. The %RMSD is computed from data acquired during a Control Cycle (CC), which is a period of time that contains at least 15 observations, generally about one month of data, but not more than 3 months.

where:

SD_cc = standard deviation
Bias = difference of observed mean from Target Value (TV)
k = statistical “coverage factor” to account for uncertainty (1 for metric, 3 to calculate specification)
TV = Target Value for the control sample (from manufacturer)

Two approaches, Two time periods for internal assessments

The RiliBÄK provide a set of explicit goals for the listed analytes. The tables also provide a concentration range for the tests - if your test value is within the specified range, then you can use the specification given in column 3 of the table. Note that multiple ranges are provided in several cases (e.g., total bilirubin, digoxin, ethanol, etc.) where performance goals at low analyte concentrations are different than for the rest of the range. If your level is outside the concentration range(s) listed in the table, however, then you must consider that the test is not on the list and use the second approach.

The second approach directs you to develop internal assessment goals for tests not found on the list, or for concentration ranges not found on the list. This requires use of the %RMSD (Root mean standard deviation) equation above, with a coverage value of k=3, and a set of data from one Control Cycle. Note: you still need an assigned target value from the manufacturer in order to calculate the bias component and to express the result in percentage units. This calculated %RMSD result becomes the Lab Observed Specification and is done separately for each level of control used in your laboratory.

Once you have the %RMSD goal, either from the RiliBÄK tables or the calculated Lab Observed Specification, you can begin internal assessment. There are two periods of time when this internal assessment should be performed. First, there should be a daily assessment of control results versus the Error Margin (the Target Value ± %RMSD goal). Your laboratory's daily control values should not exceed the respective Error Margin.  Secondly, there should be a retrospective check for Control Cycle. For each level of control, calculate the mean, SD, bias, and %RMSD from the daily control results over the entire Control Cycle. Your calculated values should not exceed the respective %RMSD goals.

If an individual daily control result exceeds its Error Margin, the method is out-of-control and results are not released until a cause is identified and corrected. You may need to run additional QC or validation in order to resume the method.  If the monthly retrospective result exceeds the goal, the method is out-of-control. You must perform the same corrective actions as with the individual control result failure, but you may also need to do more to revalidate the method. The consequences of failing consecutive retrospective assessments are more severe, requiring significantly more effort and documentation to revalidate the method, as well as potentially needing to notify the Federal authority. The lab has to inform the federal authority as soon as they may fail two times (two months) in a row.

External Assessment Goals.

The external assessment goals in the  RiliBÄK were established through the survey process as well as with the help of consultants and specialists. It is intended that the analytes and specifications in the tables will be regularly reviewed and updated, although a formal process or specific time interval has not been specified.

Laboratories must take part in Proficiency Testing or External Quality Assessment events, also known as Ring Trials, at least every six months for the mandated analytes, as a certificate of passing from the RiliBÄK is only good for six months. Depending on the analyte, however, testing may be offered more frequently, up to eight times a year.

Another unique feature of the RiliBÄK is that the goals also indicate the grading system. Roughly half of the mandated analytes are supposed to be graded against a reference method (designated in the table as RMV), the others only against the peer group median (designated in the table as TV). Part of this reflects the greater effort in Germany to establish reference methods for external QC.

So, are the RiliBÄK really good?

It's refreshing to see new thinking in the setting of quality requirements.  The RiliBÄK bring several new approaches and calculations to the field. True, not everyone will agree with or even consult these guideline. Indeed, outside of Germany there is no mandate to use them. (In a future article, we will discuss some of the RiliBÄK calculations in depth and compare them to existing guidelines). However, these guidelines certainly provide food for thought and a new perspective. The concept that the guidelines will be periodically adjusted and updated to reflect the state of the laboratory is probably the best feature of the guidelines. For too long, the regulations and requirements in the US have been set in stone. Allowing a set of guidelines to evolve should be a part of any and all future quality specifications.

See an unofficial English translation of the RiliBÄK guidelines.