Search Peter’s website:

We’d Be Likelier to Develop a Better
Flu Vaccine If Public Health Officials
Didn’t Keep Misleading Everyone
about the Flu Vaccine We Have

In late 2009, in the midst of the swine flu pandemic, Michael Osterholm of the University of Minnesota launched a research project that came to be known as the CIDRAP Comprehensive Influenza Vaccine Initiative (CCIVI). Mike is director of CIDRAP, the university’s Center for Infectious Disease Research & Policy. I agreed to sit on CCIVI’s 13-member Expert Advisory Group.

On October 15, 2012, CCIVI released its report, entitled “The Compelling Need for Game-Changing Influenza Vaccines.” The report argued that the current flu vaccine is sorely inadequate; that a key barrier to developing a better vaccine is the widespread judgment that the current one is fine; and that the main reason the vaccine’s effectiveness is so consistently overestimated is that public health officials keep saying it is better than it is.

A few days before the report’s official release, Lisa Schnirring of CIDRAP News emailed me three questions about it. This is the response I sent her. Bits of my response were included in two October 15 articles that Lisa coauthored, “Report: Complacency, misperception stymie quest for better flu vaccines” by Robert Roos and Lisa Schnirring and “A game-changing approach to investigating flu vaccines” by Lisa Schnirring and Robert Roos.

But neither article addressed a key point I made in my answers:  that public health officials aren’t just accidentally mistaken about flu vaccine effectiveness; in their zeal to encourage people to get vaccinated, they are sometimes intentionally dishonest.

Readers should bear in mind that I served on the CCIVI Expert Advisory Group, that I have worked closely with CIDRAP in a variety of capacities over the years, and that both CCIVI and CIDRAP (and therefore CIDRAP News) are directed by Michael Osterholm. Despite Mike’s hands-off policy toward CIDRAP News, the potential for conflicts of interest is obvious.

1. First of all, what was it like for you working on this? Was there anything that surprised you about the process or the actual findings?

In a January 2010 email to the newly formed Expert Advisory Group for the research he was about to launch, Michael Osterholm described its goal as “to provide a comprehensive review of all aspects of pandemic influenza vaccine preparedness and response based on the events of the current H1N1 vaccine effort, and, on the basis of that review, to provide a blueprint for future pandemic influenza vaccine preparedness and identify applicable lessons for other mass vaccination campaigns.”

Over time, the focus shifted, broadening in some ways and narrowing in others. The report that emerged this week – nearly three years later – is devoted chiefly to the need for a more effective seasonal and pandemic influenza vaccine, and what it will take to produce one.

Although I served on the advisory group, my risk communication expertise didn’t entitle me to an opinion on the technical issues that dominated our discussion. I learned far more than I contributed, and my contributions were often (rightly) seen as peripheral. Based on the content of the advisory group meetings and calls, I expected a report that would detail the inadequacies of the current flu vaccine and the process by which it is produced, and would build the case for a different process leading to a more effective vaccine.

But then the research moved in an unexpected direction, one much closer to my heart and my field. The core logic of its argument became something like this:

1. The flu vaccine we have is better than no vaccine at all, but it is far less effective than most vaccines and far less effective than the flu vaccine we need.
2. A game-changing flu vaccine is technically achievable, but there is an insufficient head of steam to develop one.
3. One of the main disincentives for developing a much better flu vaccine is the fact that nearly everybody (clinicians, investors, manufacturers, much of the public, and even many public health officials) thinks the current flu vaccine is fine.
4. Probably the principal reason why nearly everybody think the current vaccine is fine is the fact that the public health leadership has consistently said it is fine, despite research evidence over several decades to the contrary.
5. A consistent overestimation of flu vaccine effectiveness is embedded in U.S. public health decisions (for example, ACIP’s movement toward universal flu vaccination without sound scientific support). It is also embedded in U.S. public health messaging (for example, the false but until very recently widespread assertion that the flu vaccine is 70–90% effective in healthy adults under 65 … an overstatement that was frequently made even more misleading by omitting “in healthy adults under 65”).
6. More science-based flu vaccine decision-making and more science-based flu vaccine public messaging would help build the case for producing a game-changing vaccine, even while strongly supporting the existing vaccine in the meantime.

The evidentiary burden of several of these assertions is borne by Chapter 7, in my judgment the most controversial chapter in the report.

Chapter 7 does a fine job of documenting how public health – especially ACIP – overestimates and overstates the efficacy of the flu vaccine. There are really three criticisms here:

• ACIP recommendations for ever-wider flu vaccination have been grounded in claims, assumptions, and judgments that the vaccine was more effective than it actually is.
• Early on that was because good data weren’t available, but long after there were ever-better data showing that the flu vaccine wasn’t very effective, ACIP continued to speak and act as if it were – ignoring some studies, misinterpreting others, leaning too heavily on studies with big methodological flaws, relying on plausibility and expert judgment while claiming to be relying on sound science, etc.
• In their zeal to encourage vaccination, ACIP, CDC, and the rest of the public health leadership kept telling the public (often via state and local public health officials and people’s personal doctors) that the flu vaccine worked better than it works.

I was surprised and delighted to see the CIDRAP Comprehensive Influenza Vaccine Initiative (CCIVI) expand as it did. Without Chapter 7, the CCIVI report would have been a technical assessment of the inadequacies of the current flu vaccine and the feasibility of developing a better one (with detours to look at vaccine safety, public acceptance, distribution, and a few other side issues). With Chapter 7, the report is that and more. It is also a powerful indictment of the way the public health leadership and particularly the CDC’s Advisory Committee on Immunization Practices has impeded progress toward a better vaccine by overestimating and overselling the vaccine we have.

2. How do you think the report will be received? (Some of Chapter 7 sure reads like a GAO report. Lots of investigation work went into the analysis of ACIP’s recommendations.) What areas might see some early impact from the findings?

The central claim in the report is of course its claim that the flu vaccine is a lot less effective than most vaccines and a better one is badly needed.

Many in public health will find that claim difficult to embrace. But however reluctantly, I think they will embrace it. The Lancet I.D. study paved the way; in anticipation of that study’s publication, CDC stopped claiming 70–90% effectiveness in healthy adults under 65 and retreated to the much more supportable 50–70% estimate.

Now, sadly, CDC and many lower-level public health officials often provide no flu vaccine effectiveness estimate at all in their public communications, having learned that 70–90% is scientifically unsound but fearful that the more accurate 50–70% might undermine public acceptance. This is a small example of officials not trusting the public, which is a very large risk communication problem in public health. (See “Trust the Public with More of the Truth: What I Learned in 40 Years in Risk Communication.”)

The pushback against the conclusion that the flu vaccine doesn’t work very well won’t come from public health people claiming that this conclusion is wrong (which would be hard to support), but from public health people claiming – or at least feeling – that the report shouldn’t have stated this conclusion so publicly. Speaking to capacity crowds at public health conferences, Michael Osterholm has proclaimed that “We have got to finally acknowledge that this vaccine sucks in the elderly.” (See for example http://www.medscape.com/viewarticle/721159. Registration required.) He gets modest pushback and much praise. But saying the same thing to the public will strike some public health professionals as unwise and even disloyal.

If public health officials can bring themselves to accept publicly that the flu vaccine is only 50–70% effective in healthy adults under 65 in good-match years – with no better than modest-to-minimal effectiveness in the frail elderly and some other populations – I am confident that they will support the report’s call for a game-changing vaccine.

I’m not qualified to assess the report’s position on the feasibility of a game-changer, but I do think the report will carry the day on the need for a game-changer. I look forward to a serious discussion of the report’s assessment of the barriers to a game-changing flu vaccine and how those barriers can best be overcome.

There will be much more discomfort among public health professionals about Chapter 7’s claim that ACIP and U.S. public health generally have sometimes grounded flu vaccination decisions (such as the decision to expand to nearly universal flu vaccination) in faulty studies and in professional judgments about plausibility, even when there were better studies available that painted a less encouraging picture of flu vaccine effectiveness.

I think it is crucial for the public health profession to confront and correct its inclination to diverge from the best available scientific evidence when that evidence doesn’t support its enthusiasm for a vaccine. But undoubtedly the profession will find this a more difficult challenge than what’s in the rest of the report.

How will public health professionals respond?

• Perhaps the profession will rise to the challenge and work harder to curb its enthusiasm (in policy-making and in messaging) when the data don’t support that enthusiasm.
• Perhaps the profession will accept the criticism only as it applies to the specific examples in Chapter 7, ACIP influenza vaccination recommendations and widespread public overstatements about flu vaccine effectiveness.
• Perhaps the profession will simply ignore Chapter 7 and its implications, while endorsing the rest of the report and agreeing that a new flu vaccine is needed.
• Perhaps the profession’s resentment of Chapter 7 will lead it to reject the rest of the report as well, and thus reject the report’s call for a better flu vaccine.

If this last possibility comes true, even I will wish in hindsight that the report had omitted Chapter 7 … or perhaps published it separately some months from now.

The report did omit something it might have said: that ACIP and many top U.S. public health officials and experts have sometimes been more aware of the inadequacies of the flu vaccine and the inadequacies of flu vaccine effectiveness research than they were willing to tell the public. Chapter 7 claims unambiguously that flu vaccine effectiveness has sometimes been overstated. It leaves open the question of whether it has sometimes been intentionally overstated.

Public health professionals would have found the claim of intentional overstatement even harder to hear and accept than the claim of erroneous overstatement. This is because of the dynamics of guilt and shame. To the extent that public health professionals have knowingly (or half-knowingly) misrepresented flu vaccine effectiveness, they feel bad about their own deceptiveness, even though it was for a good cause. A report that told this truth would inevitably make them feel worse about themselves. Predictably, they would project those negative feelings about themselves onto the report’s authors, in the form of anger and feelings of betrayal.

Pointing out that public health officials have sometimes got it wrong will arouse much less guilt and shame, and therefore much less sense of anger and betrayal, than pointing out that officials have sometimes been less than honest would have aroused.

Thus it may have been wise for the report to evade the question of how often official claims about flu vaccine effectiveness are accidentally mistaken and how often such claims are intentionally dishonest. I urged revisions of Chapter 7 that would have addressed this question, but it may be a good thing I lost.

I believe the issue of dishonest flu vaccination hype deserves to be addressed. I believe it threatens public trust in the entire public health enterprise – not just in flu vaccination. We need a public health profession that is scrupulous about the truth at least as much as we need a game-changing flu vaccine.

But I accept that perhaps the two needs shouldn’t be addressed in the same document. Some ACIP members and other public health professionals would have felt betrayed by a report that publicly accused them of intentional hype and carefully documented the accusation. They might well have felt the sense of betrayal so keenly that they would have had trouble hearing what the report had to say about the need for a game-changing flu vaccine.

I have written elsewhere about some of the ways in which public health professionals intentionally mislead the public about flu and other infectious diseases. See for example the following articles on my website:

• Convincing Health Care Workers to Get a Flu Shot … Without the Hype (January 2009) (with Jody Lanard)
• What the CDC Is Saying about Swine Flu Severity (December 2009)
• Why did the CDC misrepresent its swine flu mortality data – innumeracy, dishonesty, or what? (February 2010)
• Further debate on whether the CDC misled people about age-specific death rates of pandemic H1N1 (May 2010)
• Vaccination Safety Skepticism: Public Health’s Self-Inflicted Wound (January 2011)
• Vaccine Risk Communication: Dishonesty Makes Things Worse (October 2011)
• Overselling Flu Vaccine Effectiveness Risks Undermining Public Health Credibility (October 2011) (with Jody Lanard)
• Flu Vaccination of Healthcare Workers: Two Risk Communication Issues (January 2012) (with Jody Lanard)
• Misoversimplification: The Communicative Accuracy Standard Distinguishes Simplifying from Misleading (June 2012)

Here is a typical example of flu vaccine effectiveness hype. It was spoken by Carol Baker, then head of ACIP, in an October 26, 2009 interview on PBS (transcript available with a link to the audio; the quoted passage is around 4:35 into the interview):

If I can just add, we do an amount of educated guessing each year in making our seasonal influenza vaccine and I think many of your listeners have heard about years where there’s a good match, years when there’s not as good a match. But here, that we know the virus; it’s been stable as Dr. Schaffner says, so it’s a perfect match. And in perfect match years, you should get at least 90 percent protection or better.

Does anyone think that in 2009 Carol Baker actually believed the flu vaccine was at least 90% effective in a year with a good match? (CDC was then saying 70–90%; the best evidence suggested 50–70% in healthy adults under 65.) Or was she knowingly overstating flu vaccine effectiveness?

And which is worse – an ACIP head who was that misinformed, or an ACIP head willing to misinform others in order to encourage more vaccination?

I think Dr. Baker was exaggerating, not misinformed – but if she was honestly misinformed, that would be truly frightening. I’d much rather think ACIP has a bad hype habit than think ACIP doesn’t know what the data say.

Compare these two hypotheses – hype versus ignorance about vaccine effectiveness – in terms of their implications for vaccine safety. If ACIP has long known that the data on effectiveness were spotty and the vaccine was pretty inadequate, but also knew that getting vaccinated against the flu was still far better than nothing (most of the time for most populations), then ACIP’s dishonesty about effectiveness doesn’t say anything one way or the other about safety. But if ACIP can’t tell a good effectiveness study from a bad effectiveness study, if ACIP really believed the flu vaccine was more effective than the data said it was, then maybe ACIP also can’t tell a good safety study from a bad safety study, and thinks the flu vaccine is also safer than the data say it is.

It’s bad enough if U.S. public health is in the hands of people who are willing to be less than candid in a good cause – to support a good recommendation with overstated science. It’s far worse if U.S. public health is in the hands of people who can’t be trusted to read the literature and know what it says.

The truth is somewhere in the middle. I have no trouble believing that many state and local public health officials believe everything they say about flu vaccine effectiveness; they get their message points from the feds and assume the science is sound. Perhaps even some ACIP members are more misinformed than dishonest, especially those for whom flu is not a major interest.

But it is inconceivable to me that this is true of the CDC representatives on the ACIP Influenza Working Group – which the CCIVI report shows has exercised enormous influence on ACIP recommendations. The Influenza Working Group is dominated by people who know and understand the literature. Any ACIP overstatements about the effectiveness of the flu vaccine or the quality of the evidence almost have to have been intentional insofar as the Influenza Working Group is concerned. If others on the ACIP were misinformed, it was their own Influenza Working Group that misinformed them.

3. How do you think this will go over with the public? It seems like the Lancet ID study went a long way to clarify the whole efficacy issue and give consumers a more honest assessment of vaccine benefits. What are the main messaging challenges health officials will have in light of this new report?

I doubt the CCIVI report will have much effect one way or the other on the public, except insofar as it has an effect on the public health profession. I expect and hope that the report will get significant news coverage in the coming days, but the direct effect will fade quickly. It’s the indirect effect that matters.

Best outcome: The public health leadership changes its flu vaccination messaging to something like this:

As you may have heard, the flu vaccine works only about 60% of the time for healthy adults, and less than that for seniors and people with some other health problems.

The paradox is, the flu vaccine still prevents an enormous amount of illness. Because influenza is so common, cutting your chances of getting it by 60% is a huge benefit, and a powerful reason to get vaccinated.

But 60% (and sometimes less) isn’t good enough; most of our vaccines do a lot better than that. We need to work harder to develop a more effective flu vaccine. In the meantime, the flu vaccine we’ve got is a lot better than no flu vaccine at all.

If this is the messaging that’s cascaded down to state and local officials and individual doctors, I think the public will get it. People will get vaccinated without being set up to feel lied to if they later learn the truth. And people might even support increased funding for a game-changing flu vaccine.

The anti-vaccination movement will of course seize on the new messaging as proof that “even the CDC admits” the flu vaccine doesn’t work very well.

It may also point out that public health officials are changing their tune only because the CCIVI report forced them to do so. And it may quote Chapter 7 as evidence that ACIP recommendations aren’t grounded in sound science.

I’d like to see ACIP and other public health leaders acknowledge publicly that these are valid criticisms. But that’s probably too much to expect. I’m more hopeful that some public health professionals will admit to themselves and each other that the criticisms are valid; that they will work internally toward more stringent scientific standards and more candid messaging; and that they will start telling the public that a better flu vaccine is a high priority.

Worst outcome: The public health leadership attacks the report, unwilling to concede publicly that the flu vaccine is inadequate and a better one is needed, far less that flu vaccine decision-making has sometimes been over-optimistic and unscientific. Flu vaccination messaging continues to be hyped (though I can’t imagine it’ll go back to 70–90%). Anti-vaccination activists cite the report not as something that “even the CDC admits” but as “what the CDC doesn’t want you to know.” The credibility of public health officials is further undermined. And the report’s plea to develop a game-changing flu vaccine falls on deaf ears.

Copyright © 2012 by Peter M. Sandman

Contact information page:    Peter M. Sandman