A common refrain in this post-antibiotic era is that we need new antibiotics and new diagnostics . While the challenges of developing a new antibiotic are better acknowledged and understood, many are very surprised that we still use fundamentally the same procedure as used pre-WWII.
So, how is it that we are, at the start of 2020 and ~30 years since the dawn of the PCR-era, still talking about the need for better bacterial diagnostics? There are multiple technical and real-world challenges that continue to stymie the development of PCR-based tests such as speed, volume that can be processed, number of bacteria identified, and so on. Sensitivity and Specificity of a PCR-based test (or rather the lack of sufficient Sensitivity and Specificity) are two key performance attributes that have held back widespread clinical use of PCR-based tests. For this discussion, we focus on identifying bacteria directly from blood for guiding treatment of bloodstream infections, especially those that might lead to severe sepsis or septic shock.
Multiple PCR-based approaches are being developed to rapidly identify bacteria from blood. To a neutral observer it would appear that the reason we do not have a suitable bacteria test is because we have not developed a good enough PCR-based test. And that suitably optimizing such a test would solve the problem. Is this correct? Or are there some fundamental limitations that have held back clinical application?
In our opinion, the fundamental technical limitation is centered around the sensitivity of PCR-based tests and a fundamental practical limitation is centered around the ability to lower costs to a level that the healthcare system can support.
There are numerous text books and reviews providing a basis for determining the sensitivity and specificity of a test. Here is one. Briefly, Sensitivity refers to how good a test is at correctly identifying people who have the disease. While Specificity is concerned with how good the test is at correctly identifying people who do not have the disease.
What can we learn from these numbers for a bacterial diagnostic based on PCR? Let’s take the case of a hospital that analyzes 100,000 bacterial ID tests a year.
The percentage of samples that turn positive upon blood culturing ranges from 5 to 10%. Using the upper end of this range it means that out of the 100,000 samples submitted for bacterial ID tests, ~10,000 turn positive. [Note: This is frequently viewed as the number of samples that actually contain bacteria though there is widespread acknowledgment that culturing underestimates the number. More on this later. But, for the moment, since culturing is the gold standard (albeit an imperfect one) let’s go with 10,000 samples out of the 100,000 samples contain bacteria and therefore represent an infection.]
Data from two commercially available PCR based tests exhibit sensitivity and specificity are shown in the table below.
| Sensitivity | Specificity | |
|---|---|---|
| Septifast ® | 42.9% | 88.2% |
| SepsiTest ® | 28.6% | 85.3% |
The sensitivity of these two tests range from 28.6% to 42.9%. In other words, if the PCR result came back negative (suggesting there is no bacteria), you could only be ~43% sure that there really was no infection! Think about that for a moment. If you were a physician making this decision, would you decide not to administer antibiotics based on this result? You’re most likely going to administer the antibiotics as you’re not that confident that you can rely on the negative result.
Similarly, the specificity of these two tests range from 85.3% to 88.2%. In this case, you can be 88% sure that if you got a positive result, the patient has an infection though there’s a 12% chance that the result may not be accurate.
Let’s apply the above numbers to our hospital that processes 100,000 blood culture samples. Suppose that these samples are tested by a PCR-based test instead of being submitted for blood culture testing. Using the Sensitivity and Specificity numbers from above, this means that, of the 100,000 samples, 15,400 will yield a positive result. But, only 4,600 of these are truly positive. So, you will end up administering antibiotics unnecessarily for a large number of patients based on the 10,800 positive results.
Similarly, you would have had 84,600 negative results of which 5,400 are false negatives. This means that you would have chosen not to administer antibiotics despite patients needing it.
| “True” Result | ||
|---|---|---|
| Positive | Negative | |
| PCR test positive | 4,600 | 10,800 |
| PCR test negative | 5,400 | 79,200 |
As a physician or a hospital, the number of incorrect decisions resulting by relying on such a test is too high – jeopardizing patient safety. Does it really provide a significant benefit over the current practice where antibiotics are administered based largely on clinical judgment? At the present moment at least, the general opinion is no. Since it does not offer significantly helpful guidance relative to current practice it does nothing to correct overuse or misuse of antibiotics.
Most of the time when we consider how to use the result from a diagnostic test, we ask ourselves, if it yields a positive result would I know what to do next? That is an important question. But, in this application, it is equally (if not more) important to ask if the test yields a negative result, would I know what to do next? There is not sufficient data (and may never be) to guide treatment decisions when a PCR-based test is negative. This is the fundamental technical limitation.
Besides their performance, PCR-based tests also create problems due to their cost and operation. A single test costs $70 – $200 based on the number of bacteria identified. Using our example, if such a test were used for each sample currently submitted for blood culture, it would cost each hospital $7-$20 million (just in consumables). Which consumes a significant portion of a microbiology lab budget leaving little to no money for other tests. And still leaves a large number of bacteria unidentified!
In theory one can design a PCR test that is multiplexed to cover a large number of bacteria. But, this will increase the cost per test. Splitting the number of bacteria identified across multiple test panels is one way of covering a broader range. But, this takes away the advantages of cost reduction due to large volumes. How does one respond to new strains or species that are of urgent concern that was not anticipated? In different regions of the world? The modifications to the manufacturing process to address these different demands make it very challenging for the manufacturer to keep costs low.
So, at a fundamental level, PCR-based tests for identifying bacteria directly from blood face limitations due to poor performance, high costs, and appear to not aid decision-making or operational efficiency.
Significantly improving sensitivity and controlling costs are therefore necessary, but not sufficient, conditions for enabling widespread use of PCR as a bacterial ID test direct from patient sample. And until they are satisfactorily resolved, there will continue to be a reluctance in using such tests to guide antibiotic treatment.
Which means that we owe it to ourselves and all patients to consider and develop alternative ID techniques. It’s not that one technique is a winner and the other the loser in such an effort. The problem is large enough that multiple solutions will be needed. The more diagnostic options we have, the better we can aid physicians in making the best decisions for treatment while slowing the spread of antimicrobial resistant bacteria.