:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/45660964/125767586.0.jpg)
If there was ever a textbook case in how to botch a health story, the Toronto Star's investigation into the "dark side of the HPV vaccine" is it. The report not only misrepresented the science behind the vaccine, but when well-meaning critics (including me) tried to point out the flaws, we were ignored, insulted, or maligned. (See our story below about what unfolded over the last two weeks, or listen to this podcast, this one, or this one.)
After the paper initially fumbled its response to public concern, the Star has since tried to correct the record. The paper's public editor criticized the story, and the publisher admitted the Star "failed" its readers. Today, over two weeks after the HPV story ran, the Star announced that the piece would be removed from the internet.
These decisions aren't easy for any newsroom to make. Retractions are rare in the news business. They are painful for the editors and reporters involved. In this case, however, the Star did the right thing.
Below is our original story about what was wrong with the Star feature.
On February 5th, the Toronto Star, one of the largest newspapers in North America and the most-read in Canada, published a story that is everything wrong with vaccine reporting in one dangerous package.
The story was, at its core, a collection of unproven anecdotes that suggested, among other things, that dozens of women north of the border had been harmed or worse by the Gardasil HPV vaccine.
A wonder drug's dark side: A Star investigation looks into the HPV vaccine Gardasil. #health http://t.co/tJolD8HCY9 pic.twitter.com/iU3bF1zNbJ
— TorontoStar (@TorontoStar) February 5, 2015
After getting the shot, one woman in the story reportedly experienced "egg-size lumps on the soles of her feet, her joints swelled and her limbs twitched uncontrollably." Even more horrifically, the Star reported, "One needed a wheelchair, another a feeding tube. A 14-year-old Quebec girl, Annabelle Morin, died two weeks after receiving the second injection of the vaccine."
These tales of suffering and death are awful. Stomach turning. But they are just that: stories. They are in no way proof of harm from this vaccine, which is why so many researchers, doctors, and public-health officials were totally baffled by the feature and took to social media to pick it apart. (This Storify by Karen Geier sums up the controversy.)
The Star's efforts might be excusable if there wasn't research available to contextualize the anecdotes. But there is data. And the newspaper alternately didn't use it or badly misunderstood it.
The reporters relied on bad data
In medicine, anecdotes are considered the least helpful type of evidence. They are biased, unrepresentative, and, as often as not, misleading.
The Star's investigation relies on just this type of evidence: anecdotes from people who believe they were hurt by the vaccine. What's worse, while the Star cherry-picked damning cases about the vaccine's alleged harms, they ignored the reams of independent studies we have involving millions of women around the world that show the vaccine is safe. This is like saying homeopathy is useful based on people's personal stories after getting the treatment instead of relying on the randomized trials we have showing it's useless.
Consider a couple of the safety studies we have. This JAMA 2009 analysis of adverse events reports, for example, shows the vaccine is no more harmful than any other, and this more recent BMJ study, involving about a million girls in Denmark and Sweden, found there was no association between the vaccine and a range of harms, including autoimmune, neurological, and venous thromboembolic adverse events. Unconscionably, the Star's report doesn't specifically mention either study.
San Francisco ob-gyn Dr. Jen Gunter did a good job debunking the Star on her blog. In an interview, Gunter said, "I think that article is the equivalent of the Oprah show with Jenny McCarthy," referring to the program featuring McCarthy on her thoroughly debunked views about the link between vaccines and autism. (To understand why those claims have been discredited, see here. It's worth noting that Andrew Wakefield, the fraudulent researcher behind the MMR-autism scare, also endorsed the Star story on his Twitter account. Quite the company they're keeping.)
The story missed important context
When anecdotes are all we have, they can be extremely powerful. Early stories about patients from doctors, for example, led to the tragic discovery that mothers who were taking thalidomide for morning sickness were having babies with missing limbs. These reports surfaced long before a randomized trial could be done — and spared thousands of babies.
But what was crucial about the reporting on those thalidomide side effects was that health researchers and doctors didn't have piles of contradictory studies involving millions of people showing the drug was safe. There, the stories were an early warning that was quickly confirmed by better evidence. The Star's story, however, is just the opposite.
To back the stories from patients who believe they were harmed, the Star reporters wrote, "In the US, where there is a public database of vaccine-related side-effect reports collected from around the world, the Star found thousands of suspected cases, including more than 100 deaths."
But the data they are referring to here is also entirely anecdotal and, as one infectious diseases doctor put it, "notoriously nonspecific."
Understanding the sleight-of-hand here requires a bit of explanation. Pharmaceutical companies run high-quality tests on their drugs — called randomized trials — for efficacy and safety before they hit the market. But because these trials only run for a short period of time, and involve thousands of people and not the millions who will get vaccines, they may not reveal rare side effects that could turn up when the vaccines are given to entire populations.
That's why, after vaccines are approved and on the market, the CDC and the FDA rely on something called "post-marketing surveillance" to monitor safety. The government has active surveillance safety networks, and also tracks reports of adverse events sent in by physicians and patients on a database called the Vaccine Adverse Event Reporting System (or VAERS). Because the latter are self reported, they may represent true reactions or just coincidences. Doctors investigate the serious ones to find out. And if they see patterns, they'll pull the vaccine or drug from the market or suspend its use.
When the Star referred to all these cases of adverse effects in a US public database, they were referring to VAERS — but they didn't mention the important limitations of the data, that they are essentially just more unproven anecdotes. The Star sold the VAERS data as if it was evidence when it is, in truth, just more stories!
This fact really enraged the British author, physician and epidemiologist Dr. Ben Goldacre. In a scathing critique of the article on Twitter, he wrote, "Reporting the raw data from an open adverse event reporting system in that manner is simply misleading, and an abuse."
@HeatherMallick @karengeier @TorontoStar @TorStarEditor I’m disgusted and appalled that you’d invoke my name in trying to defend yourselves.
— ben goldacre (@bengoldacre) February 9, 2015
@HeatherMallick @karengeier @TorontoStar @TorStarEditor You have abused that trust, with a platform so large that you will inflict harm.
— ben goldacre (@bengoldacre) February 9, 2015
We need more investigative health reporting
There is a well-documented transparency problem in medicine and health-regulatory affairs. We need journalists who can crack open stories about health the same way political reporters do on justice or defense. But this particular story didn't do that. It hinged on sensational anecdotes and ignored the best-available evidence we have on an important vaccine.
It's understandable that people have some hesitancy about vaccines. You're giving otherwise healthy individuals sometimes painful shots — that carry real, albeit extremely remote, risks — for the potential future gain of avoiding diseases.
But the fact is, we have extremely good data to support the safety and effectiveness of the vaccines we currently use, and these shots are an overwhelming benefit to public health. In the case of the HPV vaccine, it's a lot more personal, too: it's a live-saving intervention that literally stops girls from getting a deadly type of cancer.
Over the weekend, and again on Monday morning, I wrote to the Toronto Star about my concerns. The editor-in-chief Michael Cooke replied first. He dismissed the questions I raised, and pointed to a "very pro-Gardasil story" I wrote for Vox recently. He then said that my "time might be better spent doing your own Vox-paid-for research into Gardasil-good-and-questionable rather than idly picking into other reporters' work" and that I should "stop gargling our bathwater and take the energy to run yourself your own, fresh tub." He's a charmer.
The public editor Kathy English also responded, saying she was looking into the complaints about the piece. She noted that the story was part of the Star's ongoing investigation about transparency in medicine, that it "does not conclude that the vaccine caused any of the suspected side effects," and that it had included a number caveats such as the line, "There is no conclusive evidence showing the vaccine caused a death or illness."
That nuance, however, is lost in the collection of scare-mongering horror stories. And there's no doubt that, despite the caveats, this report will turn people away from the vaccine unnecessarily. This is a particular tragedy in the Canadian context, where the uptake of the HPV vaccine is already abysmally low.
English also wrote, "The story was not intended to be either anti-vaccine nor pro-vaccine." The trouble is that's not how readers will see it. And, given how little work the paper did to contextualize their stories with the best-available evidence, that's not an excuse they can really hide behind. The sins of omission are almost as bad as the sins of commission. Their story will be fuel for vaccine deniers and it will instill unnecessary fear in the hearts of parents. It may lead to people skipping a vaccine that could have saved their lives.
