The authors thank the following people for research and other assistance: Marianne Ide, Eric Ostermeier, Charles Riemann, Daniel Slotwiner, James Smartt, and Alan Yang. This research was supported by a grant from the Pew Charitable Trusts and was conducted at the University of Minnesota and at the Paul F. Lazarsfeld Center for the Social Sciences, Columbia University. The responsibility for the analysis and interpretations reported here is the authors'.

* The future of Social Security, Medicare, and entitlements more generally are once again a hot issue, and the media have provided misleading if not distorted reports on public opinion and polls.

* Journalists' references to public opinion and polls were shallow, offering little discussion of the wording of questions or even the substance of Americans' preferences toward such critical topics as the future of entitlements and the privatization of Social Security. Instead, journalists settled for generic characterizations to "poll after poll" and to vacuous references to "strong" or "weak" public opinion. Audiences are helpless to evaluate polls and public preferences on their own; they are the victims of the journalistic equivalent of a drive-by shooting.

* The tendency of journalists to present each poll as if it were a startling new discovery creates the false impression that there has been a change in the public's confidence and uneasiness toward entitlements. Stability, however, more accurately characterizes public opinion but audiences have not been presented with identical or similarly worded questions that would make it clear whether a poll continues an earlier pattern or represents an actual change.

* Journalists use polls to skirt substantive issues in favor of covering the political game -- the inside story of strategy and political gamesmanship. Forty percent of poll reports on entitlements failed to even address policy issues and instead focussed on the performance and political strategy of President Clinton and congressional Republicans.

* Journalists can avoid distorting public opinion in the future by maximizing the information given the audience on polls results and question wording; by presenting new polls in the context of results from previous polls in order to demonstrate whether public opinion is stable or changing; and by focusing on the substantive content of Americans' preferences instead of on political strategy.

Public opinion polls are a favorite tool for Presidents, members of Congress and interest groups who are intent on crafting their presentations in order to build support for their policy positions. But, polls are not only affecting the conduct of political battle; they are also influencing journalists and, by extension, Americans' understanding of public opinion.

This "pollwatch" investigates how journalists report poll results on the topics of Medicare, Social Security and entitlements more generally. We find disturbing patterns. The media's use of polls is producing shallow and myopic reports that track the political strategy of insiders rather than conveying substantive information on the public's preferences toward entitlements. Superficial and non-substantive reporting has become a staple of the fluid world of electoral politics, where candidates often practice the art of ambiguity. It is striking, though, to see a similar pattern of shallow reporting on the more weighty topic of Social Security, Medicare and entitlements more generally.

This pollwatch is based on a comprehensive search of nine prominent broadcast and print media -- the New York Times, Los Angeles Times, USA Today, Washington Post, Business Week, Time, Newsweek, ABC News, and CNN News. Our study captures the turbulent debate over entitlements since the Republicans seized control of Congress in the 1994 elections, stretching between October 31, 1994 and August 24, 1997. A systematic search identified 296 news reports of polls.

Journalists' references to polls are, not surprisingly, sensitive to the political battle. The majority (55%) of poll reports on entitlements focussed on Medicare, which Clinton and the Gingrich Congress fought over. Social Security, which Republicans studiously avoided, was the topic of 27% of poll reports. (It is perhaps telling, though, that even with the political benching of Social Security it still was the focus of a quarter of poll reports.)

Journalists loaded their stories citing public attitudes toward entitlements with shallow references to polls. Polls were slipped into stories as a quick frame of reference or hook; they were the journalistic equivalent of a drive-by shooting.

News reports persistently neglected to offer information that would indicate the genuine range of public preferences toward entitlements. Nearly 40% of references to polls offered no numbers at all. Only a quarter of the references discussed them in an indepth manner as the main focus of a story; the overwhelming majority of references cited polls in passing. In addition, polls were often briefly cited in a rapid-fire delivery. Over 80% of the references to polls consumed the equivalent of 10 lines of newspaper text or less; half of the references were 5 lines or less. It was rare for poll results to receive much genuine discussion.

Journalists settled for generic characterizations to "poll after poll" and to vacuous references to "strong" or "weak" public opinion, which left the audience helpless to evaluate public preferences on their own.

One of the media's favorite drive-by poll reference was to -- as Jim Angle on ABC's Nightline put it on November 14, 1994 -- "deep doubts about generational fairness" by younger Americans as the baby boomers prepare for retirement. A fuller discussion would have revealed that although seniors are more sensitive to threats to Social Security and Medicare, young Americans have consistently been just as supportive (if not more so) of spending on these programs. The alleged "doubts" take on a far different meaning when greater thought and time is devoted.

In addition to drive-by references to polls, news reports also persistently failed to provide information to evaluate the quality of the polls themselves. The audience was kept in the dark and denied the opportunity to independently weigh a poll's credibility. About 80% of journalists' references to polls failed to offer the wording of questions, the responses to the alternatives presented in a question, and the error produced by sampling and other attributes of the survey.

Anne Willette opened her October 1, 1996 story for USAToday by heralding a poll that purports to show public support for privatizing Social Security. "Almost six in 10 people," she informs us, "want to invest some of their Social Security taxes themselves -- even though they might end up with less money at retirement." Missing is the question wording or reference to alternative surveys that have repeatedly shown public uneasiness with privatization. For instance, a December 1996 Time/CNN poll reported that 36% favored privatization and 56 opposed it. (The poll asked: "Some people favor investing a portion of their Social Security tax funds in the stock market because this might lead to higher investment returns. Other people oppose this idea because this is too risky. What is your opinion?")

No single poll on entitlements has gotten more eye-opening attention than the September 1994 survey of eighteen to thirty-four year olds that was sponsored by Third Millennium. Journalists followed Third Millennium's lead and pitched the poll as suggesting that young Americans considered UFOs more likely than the prospect of collecting Social Security. Although journalists conveyed the impression that Americans weighed the relative likelihood of each and found UFOs more probable, the actual survey never offered respondents a direct comparison. Instead, it offered two separate questions, with the Social Security question appearing fifth and the UFO question fourteenth (it was the last substantive question before standard queries about gender and region.) Americans may well have offered different responses if they were actually asked to compare UFOs and Social Security's future. In addition, the three questions preceding the item on Social Security's future evoked negative perceptions of the program, which may well have artificially depressed confidence in the program. The bottom line is that the failure of journalists to report or apparently investigate Third Millennium's question wording produced widespread confusion about the poll and public opinion.

One of the axioms of journalism is that the story has to be "new." A story that finds little important change lacks newsworthiness.

Journalists typically convert polls into news by presenting each poll as if it were a startling new discovery. Ninety-five percent of references to polls failed to place the results in the context of previous results. The audience is simply unable to decipher whether a poll continues an earlier pattern or represents an actual change. Previous research indicates that Americans as a whole harbor fairly stable preferences. The effect of journalists' rush to crown each survey as a new finding is to exaggerate the nature of change in public opinion.

Third Millennium's UFO poll, for instance, was trumpeted by journalists as evidence of young Americans' sinking confidence in Social Security as the baby boomers headed for retirement. The implicit message was change -- American youth were losing confidence in Social Security's future. Even a brief reference to previous findings would have revealed, however, that weak confidence in the programs' future by all age groups is not new; it has been a fairly consistent pattern since 1975. The differences between younger and older Americans on entitlements have not been clearly widening with the approaching retirement of the baby boomers.

The debate over entitlements focusses on substantive policy issues: Is Social Security and Medicare sustainable over the coming decades? What, if anything, should be done? These are policy issues on which politicians and experts in social insurance disagree.

What is striking, though, is the extent to which journalists use polls to skirt the substantive issues at stake in favor of covering the political game -- the inside story of strategy and political gamesmanship. Forty percent of poll reports on entitlements failed to even address policy issues. Instead, 31% of the references to polls concentrated on the performance or popularity of President Clinton, congressional Democrats and Republicans or other political activists. (On the politically contentious issue of Medicare, nearly 50% of references were pitched to the performance of political activists.) Another 28% of poll reports emphasized the political strategy of the activists. (It rose to 37% during the Medicare wars.)

Richard Lacayo, for instance, depended on polls in his March 13, 1995 story for Time to highlight the political strategy of Senate Democrats in fighting the Balanced Budget Amendment. Polls provided the glue for Lacayo's interpretation of the Democrats' strategy to portray their opposition in the guise of defending Social Security. John Greenwald's October 16, 1995 story for Time similarly opted for polls to pinpoint the ineffectivess of Republican strategies to reform Medicare. Polls allow reporters to focus on their interpretations instead of on policy substance.

By far, the single most popular use of polls by journalists was in posing a tradeoff between protecting entitlements and balancing the budget. The effect was to narrowly present only a slice of public opinion and to portray Americans (at least implicitly) as irrational. For instance, CNN's Donna Kelly kicked off her story of the Senate vote on the balanced budget amendment on February 28, 1995 by highlighting the "mixed messages" of Americans who simultaneously supported a balanced budget and opposed cuts in Social Security and Medicare. The take home point was that politicians were tied in knots by the contradictory preferences of Americans.

While framing the debate in these terms fit the battle lines in Washington, it distorted public opinion. Americans support approaches to deficit reduction that avoid social program cuts; polls suggest that the public favors reducing defense spending, forgoing tax reduction, limiting net benefits received by the rich (including even Medicare and Social Security), and raising taxes on the rich and on tobacco and alcohol. The problem was that the public's preferred approaches were ruled out by elites. But, journalists neglected to present a robust range of public attitudes or an appreciation for the multiple and competing considerations that most Americans bring to the issue of entitlements.

Journalists distort and misrepresent public opinion. Polls were generally not the primary focus of stories on entitlements. Rather, important references to polls were typically short, shallow, and misleading. They were in effect shoe-horned into the media's standard concern with political strategy and the "inside" story.

Reporters can avoid distorting public opinion by abiding by three rules. First, maximize the information given the audience, even if space limitations make any reference to public opinion short. The content of poll reports can be improved by expanding the space devoted to analyzing polls and to presenting the questions, the results, and errors that might be involved. No consumer should trust a poll without knowing, at a minimum, its full question wording.

Second, the news media should present polls in the context of results from previous polls that used identical or similarly worded questions. This is the only basis on which consumers can judge whether a new poll represents a genuinely "new" change or simply the continuation of longstanding patterns. Finally, journalists' discussion of public opinion should focus on the substantive content of Americans' preferences. Americans' attitudes towards entitlements and other issues are complex. Instead of rushing to pigeon-hole public opinion in the political boxes that characterize divisions in Washington, journalists ought to present the multiple and competing considerations that Americans bring to their evaluations. Let policy makers wrestle with aligning their preferences with those of the public.