Americans’ responses to surveys can sometimes reveal startling ignorance and a puzzling belief in half-truth, even obvious lies, although a topic’s direct relevance to people’s daily lives matters a great deal in terms of knowledge levels. Among the most egregious recent examples of ignorance is the apparently large number of people who did not believe President Obama was born in the United States, even though it is indisputably true that he was born in Hawaii. For some, this public opinion controversy, silly at one level, seems to embody all that is seriously wrong with the country — a public drifting away from facts, slouching toward some permanently balkanized and politically polarized state. It feeds, perhaps, into a certain public/mass media obsession with conspiracy theories. However, new research analysis reveals that questionable and varying polling methods may have played a role in fanning the flames of this “birther” controversy.
Time and again, correcting misinformation has proven exceedingly difficult, as media corrections serve to perpetuate the original error and rebroadcast the misinformation. Researchers find that people often filter information through an underlying set of cultural beliefs, engaging in “motivated reasoning.” The findings of recent studies suggest some better techniques for correcting misinformation; frequently, having a credible messenger delivering the corrective — the power of particular sources — is key to reaching audiences who may not be susceptible to objective facts.
A 2014 study published in Public Opinion Quarterly, “Public Misunderstanding of Political Facts: How Question Wording Affected Estimates of Partisan Differences in Birtherism,” analyzes the structure of certain key polls in 2010 and 2011 that kept alive the misleading issue of President Obama’s origins. The study also provides results from a survey the researchers conducted in May 2011, after President Obama had released his long-form birth certificate to help settle the matter. Polling groups frequently found that there was a large partisan gap in terms of belief on the issue — that Republicans were much more doubtful of President Obama’s birthplace, sometimes exceeding Democrats by as much as 48 percentage points.
The paper’s lead authors, Jon A. Krosnick and Neil Malhotra of Stanford University, provide insight into the nature of the survey industry that should serve as a loud warning for media members: “We put a spotlight on one instantiation of a story that has played out on the national stage over and over again in recent decades: different survey organizations each craft questions ostensibly measuring the same opinion, and yet the organizations employ different question structures and different choices of words. It is almost as if each organization strives not to ask others’ questions and instead seeks to take a unique approach to measurement.”
The study’s findings include:
- The evidence suggests that “apparently very different results were partly attributable to the use of leading introductory sentences in one closed-ended question. Removal of those sentences, which were not present in other questions, caused obtained results to be more similar to those produced by the other closed-ended questions.”
- For example, in April 2011 a CBS/New York Times poll asked a closed-ended question in such a way that produced a high degree of skepticism about President Obama’s origins, particularly among Republicans. The survey question read as follows: “According to the Constitution, American presidents must be ‘natural born citizens.’ Some people say Barack Obama was not born in the United States, but was born in another country. Do you think Barack Obama was born in the United States, or do you think he was born in another country?” The partisan gap revealed by this question (48 percentage points difference between Democrats, 19%, and Republicans, 67%) was larger than that produced by the four other surveys conducted during spring 2011 by other groups.
- In an experiment in May 2011, the study’s researchers replicated the CBS/New York Times survey wording and then compared it to responses to a simpler question: “Is your best guess that Barack Obama was born in the United States or that he was born in another country?” Among Republicans, this simpler question found diminished skepticism about President Obama’s origins, by 17.6 percentage points. This experiment “illustrates the impact of leading introductory sentences and raises caution about employing them in the future.”
- One explanation for the variation is that, with the introductory sentences in place, “some respondents may have answered differently after hearing the introductory sentences because those words constituted instructions about which answer was anti-Obama. Viewed in this way, the CBS/New York Times question may have accurately tapped some people’s anti-Obama sentiment but did not necessarily accurately tap their understanding of Mr. Obama’s birthplace.”
- Overall, open-ended questions, where respondents are not forced to choose but rather must provide their own answer, “yielded the highest rates of apparent correct understanding and the smallest partisan gaps.”
The researchers conclude that “it is useful to note that these results constitute a cautionary tale about political observers’ tendency to critique the American public as lacking correct factual political information…. One can make that case based on the evidence from some of the polls done about Mr. Obama’s birthplace. Yet other polls suggest a much greater prevalence of correct understanding. Therefore, it may be important to carefully attend to question-wording effects before blaming the public for being ignorant.”
Related: For media members, the episode is a reminder that it is well worth understanding valid polling methods and asking hard questions before characterizing a given poll as an objective measure of public sentiment. For another case study in survey errors and controversies, see the 2009 Public Opinion Quarterly study, “Dynamics of Poll Performance during the 2008 Presidential Nomination Contest.”
Keywords: polling, Hawaii, Obama, presidency, birther, GOP