Now that failed inspections, government sanctions and a criminal investigation have shadowed the once stellar expectations surrounding the blood-testing company Theranos, the big question is turning from whether the company will eventually prevail to why so many people were so enamored of it in the first place.
Some cite the charisma of the company’s founder, Elizabeth Holmes, and her dream of conducting all blood tests with a finger prick. But other startup founders are charming and have compelling dreams and haven’t amassed hundreds of millions in venture capital. At its peak, the company was valued at $9 billion.
Another explanation is that falling for Theranos was a symptom of an exaggerated faith in the power of medical testing. Blood tests are no longer just a tool to diagnose disease but are seen a way to predict and prevent it.
There’s value in preventive medicine, but experts in laboratory testing say that doctors and their patients credit the numbers that come out of screening tests with more accuracy and predictive power than they deserve.
“This is not something you typically learn in medical school,” said Stephen Master, a professor of clinical pathology at Weill Cornell Medical College. “You turn on a machine and you get a number, but often we have different methodologies for testing the same thing.” Cholesterol tests are reasonably reliable, he said, but not because tests produce some perfect number. It took years of work to get cholesterol measurements harmonized to a standard. And even now they aren’t perfect.
In general, there’s more uncertainty when patients are healthy. The controversial test for prostate specific antigen had proven useful for assessing treatment success and recurrence risk in patients who’ve already been diagnosed with prostate cancer. But PSA testing has run into trouble as a way to screen healthy men. The test produces false positives that can lead to invasive treatment with nasty side effects for men who could live comfortably with prostate cancer for the rest of their lives.
These broader limitations of lab testing were, to experts, the message of a recent study comparing Theranos unfavorably to two established test companies, LabCorp and Quest. The results, published in theJournal of Clinical Investigation, cast doubt on Theranos’s claim to have revolutionized medical testing even though many of the discrepancies were modest and Theranos challenged the results.
Sixty healthy patients underwent 22 different tests. In some cases, such as the test for LDL cholesterol, a few patients got a normal result from Quest and an abnormal one from Theranos and LabCorp. These tests have big implications since they often determine whether a patient goes on a permanent course of cholesterol-lowering drugs.
Overall, the tests conducted by Theranos came out with abnormal results more often than either of the other two. The most likely interpretation, say the experts, is that Theranos has a higher rate of false positives.
One of the earliest skeptics of Theranos was the University of Toronto pathology professor Eleftherios Diamandis, who voiced his concerns last June in the journal Clinical Chemistry and Laboratory Medicine. There, he suggested that the some of the Theranos hype was based on an unquestioning faith in the predictive value of numbers.
He used his sister’s breast-cancer markers as an example. As a cancer survivor, he wrote, she gets tested periodically for a marker called CA 15.3, which is associated with recurrence.
If her CA15.3 on one occasion is 25 U/mL, and the next testing shows 24 U/ml she is having a party; if the next testing is 26 U/mL she loses sleep and panics.
In Diamandis’s view, her risk may not have changed at all, the variation being the result of natural biological fluctuations and inherent uncertainty in the measurement. He also notes that screening results are far from perfect predictors of recurrence. Sometimes people never get a high reading and their cancer still comes back.
He said that while Theranos won’t release details about how their technology works, investors could have at least demanded data. Some years ago he wrote to Holmes and suggested she offer free testing to 20,000 people as a trial. That way she could see how many diseases were caught early, how many diseases were missed, and how many times false positives occurred. She didn’t take his advice.
Professor Norman Paradis of the Dartmouth College medical school has also questioned not just the claim that Theranos can produce cheaper, faster, less scary blood tests, but also the assumption that succeeding in this quest would improve public health.
In a piece titled, The Rise and Fall of Theranos, which ran in Scientific American and the online magazine The Conversation, he questioned the value of the Theranos promise to run dozens of tests on a small amount of blood. “From a clinical perspective, this was always concerning, as such a shotgun approach to medical testing is actually very bad medicine.”
It’s not that he’s against blood tests in the right context. “I’m very big on ordering tests,” he said. “But I don’t immediately say we need to start treatments.” What’s needed for better preventive medicine isn’t just more tests, but more accurate tests, and a better understanding of what to do with the results.
If any good comes out of all this, said pathologist Master, it would be a better understanding of the need for thoughtful interpretation followed by careful decision making. “Medicine,” he said, “is more than getting a number out of a box.”