Healthy Skepticism A Shot of Fear

Instead of urging vaccination for everyone age 50 and older, as they had been doing, government officials recommended shots only for people 65 and older, and those in selected high risk groups. The public's response was predictable: People were upset and confused. Our local television news played a story in which a pharmacist was called "a murderer" when his vaccine supply ran out. Ironically, the crisis mentality led some to engage in behaviors that probably increased their risk. Frail elderly people, some with oxygen tanks, stood in long lines in the cold, waiting for the vaccine. Others crowded clinics and doctors' offices, increasing their chance of exposure to flu and other infectious agents.

With uncertainties about this year's vaccine supply, the CDC again recommended that highest-risk people get priority for flu shots, at least until late October. But last year's flu season may have left people confused about essential points: Just how risky is the flu? And just how effective is the vaccine? The answers to these questions may surprise readers.

How Risky Is the Flu?

First, a caveat: The risk calculations we analyze here describe typical flu seasons only. We don't consider here what the picture would be in the event of a deadly flu pandemic -- a worldwide outbreak of a new, highly virulent flu strain, the potential for which has recently drawn considerable media attention. No one really knows how likely such an outbreak is, but the risk profile would certainly change. A pandemic is a fundamentally different situation: The risk of death would be substantially higher, and untested strategies (including new treatments, quarantine and a new vaccine) would need to be implemented rapidly.

We deal here with what is known about typical flu seasons, based on data that form the basis for the federal government's flu-risk figures.

By choosing to highlight the annual number of flu deaths, the CDC employed an attention-grabbing tactic often used by public health and disease advocacy groups. It's a tactic readers should be inoculated against if they want a clear picture of the risks they face. (See "Research Basics: Understanding How Big a Risk Is," right.)

In fact, it is very difficult to know how many people die from any given disease because there is often much uncertainty in determining the cause of death. This is particularly true for the flu. That's because it shares symptoms with so many other diseases, and because people most likely to die a flu-related death are also at high risk for many other causes of death.

Flu deaths are probably undercounted because doctors do not routinely test for the flu, and because some deaths that should be attributed to the flu are given other diagnoses. For example, someone who dies from a heart attack because they are debilitated by the flu might not get counted as a flu death. Some overcounting of flu deaths also occurs: Clearly not all winter pneumonia deaths are caused by the flu.

According to the CDC, 90 percent of flu-related deaths occur among people age 65 years and older. Based on this information and the age distribution of the population, the chance of a flu-related death for people in that age group is about one in 1,000. Another way of saying this is that the chance of not dying from flu for those 65 and older is about 999 out of 1,000. (For context, the chance of a flu-related death is slightly lower than the chance of dying from a fall or other accident.)

For people younger than 65 (including children), the chance of a flu-related death is much smaller -- about one in 100,000. Of course, adults and children might be concerned about flu-related problems besides death, such as being hospitalized or just suffering with unpleasant symptoms (typically three to seven days of fever, muscle aches, headache, weakness, dry cough and runny nose). As you might guess, counting the number of flu-related hospitalizations or the number of people experiencing symptoms from the flu is even more difficult than counting flu deaths.

How Good Is the Vaccine?

Getting a shot does not guarantee you will not get sick from the flu or die from it. Recently, the Cochrane Collaboration, an international group that evaluates the evidence for various medical interventions, reviewed the medical literature on the effectiveness of the flu vaccine in preventing death.

Unfortunately, the evidence on how well the vaccine works to prevent death in the elderly is limited. Few of the existing studies are randomized trials -- considered the gold standard for medical evidence. Instead, most data are from observational studies -- studies in which scientists simply count up outcomes (here, the number of deaths that occur among people who did or did not get the vaccine).

But drawing conclusions about cause and effect from such observations is fraught with problems.

For example, a 2003 study published in the New England Journal of Medicine observed that the flu vaccine was associated with a 50 percent reduction in the overall death rate (that is, death from heart disease, stroke, cancer and all other causes combined). To attribute an effect of this magnitude solely to the flu vaccine is ludicrous: Flu-related deaths make up less than 2 percent of all deaths. If the claim were accurate, the vaccine's power would dwarf that of any other medical intervention. There is, however, a much more likely explanation: People who choose to get a flu shot are much healthier -- and therefore already at much lower risk of death -- than people who do not.

Only five randomized trials have examined the effectiveness of the flu vaccine. In these studies, patients were randomly assigned -- a selection technique equivalent to the flip of a coin -- to get either a flu vaccine or a placebo injection. But none of these studies looked at whether the vaccine prevents death. Instead, the scientists measured who developed a flu-like illness. For a summary of the findings of these studies, see "How Well Does the Vaccine Work in the Elderly?" below.

In the absence of good randomized trial data, it is still possible to gauge the effectiveness of vaccination by looking at time trends in flu vaccine rates compared with flu-related deaths in the elderly. As more people get vaccinated, you would expect the flu-related death rate to decline -- if the vaccine is effective. But, as the graph below, titled "A Windening Gap," shows, despite a dramatic increase in vaccination among the elderly, deaths from the flu and pneumonia have hardly budged. (The calculations have taken into account the aging of the population.)

For younger adults, flu-related death is so rare that it has not been reliably studied: Doing so would require a trial of millions of people.

Of course, the flu shot may have benefits besides reducing the chance of death. Many may get flu shots simply to avoid getting sick. The Cochrane Collaboration identified more than 20 randomized trials addressing this question. The overall chance of developing "clinical" flu (we'll explain in a minute) was 19 percent in those chosen, again by chance, to receive the recommended flu vaccine vs. 23 percent in the control groups.

The careful reader may notice that these percentages are substantially higher than those reported for the elderly. (See "How Well Does the Vaccine Work in the Elderly?") This is because clinical flu is defined as a set of non-specific symptoms including fever, cough and muscle aches -- symptoms shared by many non-flu illnesses like the common cold. These non-flu illnesses may be especially common in younger adults because of their exposure to other people, particularly children. To try to isolate the effect of the vaccine, scientists sometimes use laboratory tests to confirm the activity of flu virus in the blood. Using this measure, the chance of flu in the vaccine group is 2 percent vs. 7 percent in the control group.

Studies have also measured another outcome: how vaccination affects days lost from work. On average, there are about 0.16 fewer days lost from work per person vaccinated. Another way of saying this is that about 5 percent of those vaccinated avoid missing about three days of work because of the flu. (That is, 0.16 days divided by the 5 percent who benefited from vaccination equals 3.2 days.) The other 95 percent vaccinated got no benefit.

Take-Home Messages

To promote vaccine use, many in the public health community have overstated the risk of flu-related death and the effectiveness of the vaccine in preventing it. While the flu vaccine may have some important benefit (less flu-related illness), we really do not know whether it reduces the risk of death. For younger individuals -- for whom the chance of flu-related death is extremely small -- any death-protection benefit can only be very modest (and it is unlikely we will ever reliably know whether it even exists). However, we do know that the vaccine reduces the risk of being sick and time lost from work. But because the effect is small, individuals will have to judge for themselves whether it's worth the bother.

We are not suggesting that Americans forgo flu vaccines. We simply want to help people make informed decisions.

For many people, getting the vaccine is a reasonable choice. And many may reasonably choose not to get it. (Consequently, the use of flu vaccination rates by Medicare and others to measure health care quality probably does not make sense.)

Regardless, public health officials should not exaggerate risks or benefits to promote vaccination. Exaggeration carries a price: Not only do some people get scared and engage in behaviors that increase their risk (like waiting in a crowded clinic for a flu shot). They may also grow cynical and end up ignoring health messages that really matter.

Steven Woloshin, Lisa Schwartz and Gilbert Welch are physician-researchers in the VA Outcomes Group in White River Junction, Vt., and faculty members at the Dartmouth Medical School. They conduct regular seminars on how to interpret medical studies. (See http://www.vaoutcomes.org.) The views expressed do not necessarily represent the views of the Department of Veterans Affairs or the United States Government.

To respond to this article, send e-mail to health@washpost.com.