Chevron icon It indicates an expandable section or menu, or sometimes previous / next navigation options. HOMEPAGE

Here's why the jury is still out on Theranos

Medical diagnostic and blood testing company Theranos has had a rough couple of weeks, and people want to know how things are going to shake out.

Advertisement

It's time to take stock of what it's going to take to settle the debate about whether the controversial startup is for real or not.

Elizabeth Holmes, the 31-year-old founder of the $10 billion company that promises a better way of doing blood tests has finally said that in addition to seeking FDA approval for the tests that have long fascinated the tech and media world, Theranos will publish data that shows how its theoretically revolutionary blood tests compare to those run on traditional machines.

Elizabeth Holmes Theranos thumbnail
Skye Gould/Tech Insider

That is what medical experts and the science community have repeatedly asked for. As an April investigation by Tech Insider found, researchers have long been skeptical about the hype surrounding Theranos, since the company has up until now been unwilling to show how its blood tests work.

A brief recap

The latest development comes after an Oct. 15 bombshell investigative report in the Wall Street Journal alleged that according to former employees, the company's innovative technology wasn't working as well as they hoped and claimed. One former employee even alleged in the Journal report that Theranos may have manipulated the proficiency-testing process, which could be a violation of regulatory requirements. (The company maintains that the reporter misunderstood and mischaracterized their process.)

Advertisement

Holmes appeared on television and at a Wall Street Journal conference to contest the article. Theranos fired back with strongly worded responses, including a 6,000-word document meant to contest the specific points of the article that asserts that John Carreyou, the WSJ reporter, "got it wrong." The Journal stands by Carreyou's work, and published a statement saying "our articles about Theranos were thoroughly reported, fair and wholly accurate."

After the controversy came to a head, Walgreens, home to 41 Theranos test centers, announced that it would hold off on opening any new Theranos locations until "questions about its technology" were resolved.

This morning, in response to a Freedom of Information Act request, the FDA published the reports detailing its inspection of Theranos' main lab. The FDA strongly criticized the company for not responding to customer complaints, for problems with lab protocols, and for using unapproved devices.

That's brought us to now, where Holmes says that the company will finally publish data showing the validity of its technologies — though the timeframe for that is unclear. 

Advertisement

Most people, understandably, don't know what to think at this point.

But until scientists in the field can examine how Theranos' tests compare to those run by other companies — or until the FDA comes out and approves all 130 tests the company has submitted — the jury will still be out on whether or not they're as effective as Holmes and the company claim.

Here's why.

What's the verdict on Theranos?

We still won't have an answer until the data is out in the open.

Everyone wants to know if the WSJ story and the backing off by partners mean that Theranos' technology isn't as revolutionary as people originally though. What does all this mean for the company's murky future?

Advertisement

We can't know for sure. They've long said they have data that proves their tests work just as well as traditional ones, but until now — and we're still waiting for the company to deliver — they've declined to publish or share it (we've asked). We'll be left in the dark until we see a peer-reviewed comparison or until the FDA comes out and approves all the tests they are evaluating — not just one. (The lab inspections don't offer information on how reliable the test results are.)

The uproar over this scandal "just kind of confirms the reason that a lot of us are skeptical," says Dr. David Koch, president of the American Association for Clinical Chemistry and a professor at Emory University.

We spoke with Koch in April and spoke with him again after the WSJ story broke to get his reaction to these developments.

"It seems like their testing is nowhere near ready, and they are still are struggling with things," he says.

Advertisement

"There's still no publication out there validating their claims, which is a big deal if you want to gain acceptance in the regular clinical community," says Dr. Jerry Yeo, a professor and director of Clinical Chemistry Laboratories at the University of Chicago, who we also checked in with again, after originally hearing about his concerns in April.

"If you don't show data, which scientists will believe you?" Yeo asks.

In theory, Theranos is starting to hear that message now.

Why has the scientific community been so critical?

Most scientists say it's "impossible" to know without comprehensive data how accurate the company's tests are. 

The company has repeatedly claimed that FDA approval is the "gold standard" for proving that something works, but without that approval and without seeing the data, the scientific proof is notably absent.

Advertisement

Theranos likes to point out that they have one FDA approved test for the herpes simplex 1 virus, and that this approval "provides independent validation of the ground breaking Theranos System upon which the HSV-1 test is run."

theranos finger-prick
A Theranos technician performs the finger-prick test on TechCrunch's Jon Shieber on stage at TechCrunch Disrupt SF 2014. YouTube/TechCrunch

But there's a problem with that statement. The company has used that approval to claim that both its overall "system" and its proprietary devices, like the nanotainer — a small container used to hold blood — have been validated by the FDA.

It's likely that the FDA will find no problem with devices like the nanotainer, as the company has pointed out, but we won't know for sure until the device is fully approved. And at least right now, that device is not approved, as investigators underscored in the August inspection report. 

Even if their proprietary containers and tubes end up being fine, that one FDA approval doesn't validate the results of still unapproved tests. The tests are not just the the collection devices; they are also the way the collected blood is analyzed and processed using Theranos' proprietary technology.

Advertisement

The latest reports from the FDA confirm that the company has not yet received a "good-to-go" from the FDA.

Yeo says that the herpes test is most likely something called a qualitative immunological test — something easier to pull off with a minuscule amount of blood. But other tests where an exact level of a substance in blood need to be measured are far more difficult, as they require higher sensitivity.

But whether or not Theranos' herpes test works that way, there's no reason to think that the same method can be used to measure vitamin D levels or other factors in blood. Since other tests don't work the same way, it's not reasonable to argue that the FDA approval of one test "provides independent validation of the ground breaking Theranos System" — unless you just mean validation of the system for one herpes test. The company offers more than 240 tests on its web site, and Holmes has said they've submitted 130 tests to the FDA so far, which are most likely the ones they want to run on proprietary equipment.

Theranos' spokespeople have consistently pointed out that the company sought out FDA approval even when it wasn't required to get it. But if Theranos wants people to put their health on the line by trusting in these tests and so far hasn't offered proof that they are effective, they shouldn't be surprised that people in the scientific community will now wait for that approval and not just take it on faith that the system works.

Advertisement
Elizabeth Holmes
Getty Images

Have Theranos' responses reassured skeptics?

No. Concerns about a lack of evidence are still unaddressed. If the company really does release their data, that would change.

Holmes has said that critics are wrong because "people don't understand what we actually do," pointing out that they run one test on proprietary technology for now while they wait for FDA approval.

But many of their responses to criticism have avoided the actual issues. When the Journal's story alleged the company wasn't using proprietary technology to run blood tests, the story referred to the proprietary machines with an internal name the company had used, calling them "Edison" machines. In response, Theranos said that people confuse one piece of proprietary technology for another, saying the company doesn't use the "Edison" machine anymore and that tests are done on a different proprietary machine.

This is nitpicking. People looking at the company from the outside don't care whether the tests are run on an "Edison" machine or another machine named "Fermi," "Galileo," or "Turtle," as one astute commenter pointed out on an annotated version of Theranos's response on the site Genius. Observers just want to know that the proprietary machine works, whatever it is called.

In their response, Theranos implies that criticisms of their technology come from their competitors. They compared questions raised by the WSJ about the fingerstick process to a quote by Quest Diagnostics, one of the massive lab testing companies that leads the industry Theranos aims to disrupt, in another story about the company. They say comments from anonymous experts are questionable. In a TV interview, Holmes went even further, saying that their competitors were seeding the press with negative stories about the company.

Advertisement

Those companies might want to eliminate competition, but that doesn't mean asking questions about Theranos is unfair — or part of an orchestrated plot. Many people have raised questions about the company. Reputable scientists in the field have gone on the record expressing their skepticism.

The company does have respected partners that casual observers assume must have vetted their technology.

But many of those partners seem to have their own questions.

Walgreens has an interest in having reputable technology in their pharmacies, and they are holding off on opening new locations. The Cleveland Clinic is a highly respected institution that Theranos announced a partnership with earlier this year, but a representative for the Cleveland Clinic tells Tech Insider that although they are deferring calls to Theranos for this story, they are still in the early stages of their partnership and have not yet used Theranos's technology. Pharmaceutical companies that were cited by the press as Theranos partners have said they do not have ties to the company.

Advertisement

These partners indicate that there's something real or disruptive behind Theranos' tech, but we still don't know how real or disruptive it is.

theranos review 3
Melia Robinson/Tech Insider

How did Theranos become such a big deal?

The narrative that got Theranos so much attention spun out of control. In the end, the results are all that matter.

The astounding thing for many observers here might be that a health company was able to generate so much buzz before they'd proven how effective they were.

It's hard not to think that part of the cause might be the hype in the tech world itself, where it's possible to receive a $10 billion valuation without people knowing how your technology really works. The company has an illustrious board full of influential ex-government officials and corporate leaders, but few actual scientists or health experts.

The media and tech world was wowed by the story of the young Stanford dropout who was trying to change the world.

Advertisement

If Theranos just talked about a nicer blood test experience instead of a breakthrough technology that would revolutionize medicine, there would be far fewer questions. Currently they seem to be doing just fine testing blood using traditional machines. My colleague Melia Robinson recently had a blood test done at a Theranos center using a venous blood draw — with a needle — and said she was "blown away" by the spa-like experience.

But you don't get a $10 billion valuation by offering only a nicer blood-testing experience.

Some investors say they took a look at Theranos but weren't convinced they had something, so they didn't invest. GV (Google Ventures) founder Bill Maris told Business Insider's Jillian D'Onfro that "We looked at it a couple times, but there was so much hand-waving — like, Look over here! — that we couldn't figure it out," he said. "So, we just had someone from our life-science investment team go into Walgreens and take the test. And it wasn't that difficult for anyone to determine that things may not be what they seem here."

Other investors have been quick to defend Theranos and said they didn't like to see the company being criticized because "new tech is hard." But as Business Insider's Matt Rosoff points out in a piece of analysis that you should check out, "Every faker, every charlatan, and every company whose product just isn't good enough to win is taking money that could have been invested in other companies that have a better chance. "

Advertisement

And in this case, it's not just about other companies that could have better used that money.

A company that's helping people make decisions about health needs their technology to work. No one should take their word on faith, or believe it based on customer reviews. Former Apple executive Jean-Louis Gassée described test results from Theranos that he says varied significantly a few days apart — tests that Koch told us should not vary within that timeframe. Wrong and varied results could cause serious harm to a patient if those results lead them to avoid treatment or to receive unnecessary treatment. 

If Theranos is finally willing to show their technology works by publishing peer-reviewed studies in light of this criticism, it would go a long way in relieving the concerns that skeptics have. They've continually said they consider FDA approval the gold standard for showing their technology works, and observers would love to see what the FDA says.

"If they get the FDA approval then they can start using their own devices and methods," says Koch. "Without it, it’s hard to judge anything," unless there is data that can be reviewed, he says.

Advertisement

So until we see that data or the FDA comes out with a decision on the rest of the 130 Theranos tests under review so far (with more to come), we won't have a final answer.

As Koch says, "getting quality results is really the bottom line."

Advertisement
Close icon Two crossed lines that form an 'X'. It indicates a way to close an interaction, or dismiss a notification.

Jump to

  1. Main content
  2. Search
  3. Account