A Biased View of Reality?

November 23, 2010, 3:54 PM GMT+0

I've got more bad news about what people know.

In two previous blog posts, I wrote that survey respondents were, on average, quite mistaken about the proportion of illegal immigrants in their state, as well as the proportion of their state making over $250,000 per year.

It seems that Americans are similarly confused about other demographics as well. I asked respondents in this nationally representative survey about the presence of several different groups, in both their state and city. As bad as misperceptions are about illegal immigrants and the wealthy (overstated by and average of 13 percentage points and 11 percentage points, per state, respectively in the survey), there are other demographics for which reality is even more distorted. For example, Hispanics were over estimated by almost 15 percentage points (the average state is 10%, while the average survey respondent said 25%), while African Americans were overestimated by over 16 percentage points (11% in reality, 27% on the survey). However, the greatest distortion is in the number of food stamp recipients, where people are off by over 20 percentage points (11.6% in reality, 32.2% on the survey).

It appears that Americans have a very distorted view of reality. Part of these results could just be that people are just carelessly responding, without putting much thought into the questions - indeed the standard deviation on the responses is quite high - however, I don't think that careless response explains all the inaccuracy because respondents seem to be putting some thought into their responses. For example, I asked what percent of their state voted for Obama, and, on average, individuals said 51.7%, which is pretty close to reality. So, respondents do appear to be thinking about the question and trying to get it right. Also, they are not blindly giving the same answer to all questions -- for example, while they said that 16% of their state were illegal immigrants, they said over 27% was African American. Both these answers are very wrong, but they are very different guesses, indicating that people are putting some thought into their answers. And these numbers are not just a result of respondents not having a good sense of percents: they overestimated the average income in their states too, which is a raw number - although not by as much (they overestimated by 45%).

Why does any of this matter? I think it might show something systematic about the way in which people perceive their world. The phenomenon of "Wisdom of Crowds" means that a group of individuals can usually, on average, arrive at the right answer when guessing at unknown quantities: for example, guessing at the weight of a pig at the county fair. The reason for this is that the error in one person's guess is probably balanced by the error in another person's guess. If you have enough people guessing, on average, the errors should balance out. However, if there is something systematically biasing the errors this doesn't work: for example if it is immediately before lunch, so everyone at the fair is really hungry and thinking about how much they want a pork sandwich, so they overestimate the weight of the pig. It appears that there is something systematically wrong about the way in which Americans understand demographics. I am not sure about what causes this systematic bias - it might be something about the way that the media portrays the world or it might be something about the selective reception of information caused by people wanting information that reinforces the assumptions of their political philosophy. It might also be that political leaders promote a certain image of the United States in order to promote their agenda. However, if this is true, the political "direction" of this bias is not even clear - for example, I am not sure if Americans see their country as richer or poorer than it actually is - after all, they overestimate both the average yearly income and the number of people on food stamps.

Perceptions of demographics probably matter for public policy, for example the current focus on illegal immigration, when the problem is smaller than most people think it is. Because these perceptions matter, it would be good to find both the sources of the bias, for example, is it correlated with a certain political ideology, and to understand how the bias effects policy support.

(One note is that maybe if I had enough respondents per state, that respondents, on average, would be correct about their own particular state - however, I seriously doubt it. I doubt it because for every question, the bias is in the same direction for almost every state - e.g. people in every state overestimate the average income, rather than people in some states overestimating and people in some states underestimating.)