Trends

If you have a method with no regulatory approval, no validation data, from a company with no history of diagnostic manufacturing... This is Not a test.

As the first wave ripples through the US, it’s worth taking a breath to consider what comes next. Our recent past is discouraging. Crippling mistakes were made at the outset of this pandemic, ones that we are still paying for now. For lack of testing, we lost the first battle to prevent the pandemic from establishing itself on our shores. For lack of testing, we lost the ability to get a real sense of who was infected and who wasn’t. For lack of testing, we needed to burn up huge stores of PPE to protect ourselves from every unknown status.

But the initial panic is over – we hope – and now we have tests both in PCR and serology that, to some extent, for better or worse, can detect the SARS-COV-2 virus and its antibodies. Testing supplies are short, but the utter desert of testing is over. Moving forward, labs will still struggle with supplies, but that will be a sporadic battle, with moments of famine, followed by feasts of testing.

See even more stories about COVID-19 Laboratory Challenges...

One battle we CAN win is selecting the right test method.Think of this stark contrast: to our North, as of May 18th, Health Canada has only authorized two serology tests. While here in the US, we have more than a hundred manufacturers in the market, most without any proof of performance. Even those with EUA have only had to provide scant validation data in order to earn that letter. While the utility of serology testing is still debated and studied, we have a more mundane challenge: how to sort the wheat from the chaff.

The FDA, on May the 4th, finally provided a performance specification: 90% sensitivity or PPA and 95% specificity or PNA. That will eliminate a huge swath of contending test methods from consideration. Taking a look at one preprint of a very well-construct analysis of 9 Lateral Flow assays, none of the methods could meet the FDA benchmark. Even some of the methods from the major diagnostic manufacturers do not meet this goal.

In the UK, the bar has been set even higher, 98% sensitivity and 98% specificity.

This is the concrete step you can take today: eliminate from consideration any method that doesn’t meet this benchmark (you can choose UK or US, if you wish). If you have a method now that isn’t meeting this goal, work toward finding a better alternative.

Why are these performance specifications being set so high? Given a low prevalence of disease in the population as a whole, a higher specificity produces fewer false positives, thereby providing a higher confidence that a positive test result truly identifies a patient with disease. With the expectation of widespread antibody testing for surveillance of immunity, false positive subjects may be encouraged to get back to work, which may allow them to become new vectors of disease.

If you have a method that generates more false positives than true positives, more false negatives than true negatives, or generates more uncertainty rather than less, This is Not a Test.

COVID19 is particularly insidious as a pandemic. Patients are most infectious when they are least symptomatic. We are best at shedding the virus when we are least aware of being infected. So the detection ability of a test is crucial. Unlike the previous SARS outbreak, temperature checks are of limited use. We need better testing.

This is the concrete step you can take in the near future: building a system of testing that will improve the right diagnosis
One emerging trend is the need to create reflex protocols for testing. Buried in the back of the White House plan for opening up is the recommendation to use more than one test to diagnose COVID-19. If we are stuck with poor tests, or less than ideal tests, we can still get a better result by testing with more than one method. A preliminary positive can be retested to determine if this is a true positive or false positive. Likewise, a confirmatory test can separate false negatives from true negatives. Of course, the effort and expense of testing a patient twice – think of the PPE resources that must be consumed – mean that the downstream impact of a poor test method is severe ( a good example of external failure-costs that accrue outside the lab even though the failure is inside the lab). Using an “expensive” test that has higher sensitivity and specificity is a lot better then having to use two cheap methods.

This is nicely summarized by some comments from Dilbert on 5/7/2020. In reponse to the question “Is this data accurate?”, Dilbert replies “You don’t go to war with the data you need, you go to war with the data you have.” Which leads to another question; “Did you must make it sound noble to use bad data?”, which Dilbert then answers “And heroic.”

We have begun this fight in a situation where there has been a lack of good data, but going forward the data will be better because the FDA is imposing more demanding requirements for manufacturers to better document test performance AND also demanding better performance in terms of clinical sensitivity and clinical specificity. Going forward (after the first week in May, 2020), we should make use of the better data that will be available to ensure we provide higher quality testing.

Finally, here’s the bigger challenge, the long term steps you will need to take.

There’s a lot of scrambling right now over the business of testing – or the lack thereof. Reimbursement codes and revenue flow are an unavoidable issue in the US healthcare system. Many labs in the US have seen major declines in their testing volume, in their revenues, and may be faced with the pressure to layoff, furlough, or cut their staff. Precisely at the time we need to be gearing up for more testing.

There is an urgent need for more testing, not less. And under Virus Economics, there is no return to business as usual, to the normal elective procedures and lucrative practice of medicine UNTIL the virus is defeated. The first wave might not be done with us yet, but without a vaccine, there will be more waves.

If you are not already working in a regional network, informally or formally, you need to reach outside your walls, beyond your network, and coordinate all the institutions in your city, your county, your state, and even your region. At least keep track of your testing volumes, positivity rates, and resource challenges. Pool them when possible – bid together rather than against each other in your quest for supplies, and shift supplies to hotspots as needed.

If you aren’t at the table where decisions are being made, demand a seat. If you are seated with the decision-makers, demand a stronger, if not leading, role. Demand the budget you need to meet the testing needs of today, the future testing needs of your predicted apex (if it’s still coming), and the apex of the waves that may come.

If you are waiting for someone else to come up with a plan, for someone else to show leadership, for someone else to take responsibility for getting the best results out to the patients, your wait is over.

Your time to lead is NOW.

This is not a test.