If you want a snapshot of how our President’s approval rating is doing, you might ask Gallup. Gallup will tell you that just now the President is down 2%, 45% – 47%. Yet if you ask Rasmussen, they will say he is down 6%. The Economist puts him down 9 points, Fox News says -10, and Reuters/Ipsos claims he is down a full eleven percentage points. Meanwhile, Quinnipiac’s most recent poll agrees with Gallup at -2. So which pollster should you trust?
Step One: Look at Polling Averages
The first step towards not looking like a complete fool when you are shocked to discover that the candidate of your choice lost an election you swore he was winning by five million votes (Harrumph… Karl Rove…) is to make some effort to consider more than one polling source. A number of reputable web sites compile blind polling averages that are far more accurate than most individual polls. I personally like RealClearPolitics.
Step Two: Ignore All Internal Polling
Internal campaign polling is always biased. A campaign usually has two objectives in conducting polling: to create favorable numbers which can then be released to the media, and to create accurate numbers that can spur the campaign to fix whatever is holding them back. It is obvious why intentionally biased polling should be biased, but intentionally accurate polling is also usually biased. Nate Silver wrote an article that examines internal Romney campaign polls that were never released to the public and finds that every single one of them was biased towards Mitt Romney. The average bias was 4.7%.
Step Three: Adjust Numbers from Rasmussen
If you just look at polling averages, you will probably be fine. However, in the off-season (right now) or in races where polling data is scarce, these polling averages can still lead you wrong. So I am going to share a few tips that I have used to make my interpretations of polling data more accurate.
First, some pollsters have a lot more noise than other pollsters. Numbers from small, local pollsters that only conduct one poll, that have no national reputation, and that you haven’t heard of before are great as a part of a large sample of polls, but individually should be taken with a grain of salt. Pollsters that do not fit in this category include Quinnipiac, Rasmussen, Gallup, and PPP. These pollsters generally take a relatively larger sample size and conduct polling constantly, so their numbers tend to accurately reflect the ups and downs of the political world relative to previous surveys. Some people may claim that various pollsters are biased in one direction or the other, but I have found Quinnipiac and PPP to be fairly stable and reliable.
Rasmussen and Gallup are also consistent pollsters, but both showed methodological flaws in the last election cycle that systematically favored Republican candidates by several points. Gallup has revamped its methodology and has seen its polling numbers move back towards the center of the pack. Rasmussen, however, has yet to admit that its polling numbers are inaccurate. Nate Silver has written articles for the New York Times on his famously accurate page, Five Thirty Eight, criticizing Rasmussen’s polls for favoring Republican candidates by a stunning average of 4% in all of the 2010 and 2012 elections. Electoral-vote.com now offers its analysis with two separate pages, so viewers can choose whether they want Rasmussen data included. Considering the consistency of this R+4 result, however, I would not simply exclude Rasmussen from any data set, as has electoral-vote.com. Rather, I would simply adjust the numbers D+4, check to see that it is consistent with the other major pollsters, and consider it accurate. Considering our example of Obama’s approval rating, adjusting Rasmussen’s numbers D+4 yields exact agreement between the three major pollsters who have polled the issue in the past month. Certainly, it is possible that Obama’s true approval rating is worse than -2%, but my money says that that estimate is a good bit more reliable than Fox News’ -10%.
Case Study: Generic Congressional Vote
Let’s flip over to RCP’s polling average for the generic congressional ballot. Just now we are looking at D +1%. Dig in a little deeper, and we find that this number is generated by averaging three weighted polls from Rasmussen (tie), Quinnipiac (D+4), and Democracy Corps (R+1). My first question is, who is this democracy corps? I glance at their polling sample size: 841. It seems a bit small. When I look at Quinnipiac, I am greeted with almost double at 1468, and Rasmussen takes the crown with 3500.
My first adjustment is to adjust the Rasmussen number to D+4 and compare it to the other major pollster with a reputation for accuracy, Quinnipiac. What do you know? A perfect fit. At this point, I am dismissive of the unknown entity with the tiny sample size. However, I am now basing my knowledge on only two polls.
But RCP provides additional information. If you scroll down, you notice that Rasmussen has actually been conducting surveys every single week. If we expand our time window to 6/28 – 8/26, and consider every poll conducted in that time, we have a grand total of six Rasmussen surveys and two Quinnipiac surveys. Adjusting each Rasmussen survey D+4 yields the following numbers: D+4, D+4, D+1, D+5, D+6, D+3, for an average of D+3.83, which is similar to our average of D+4.5 from Quinnipiac, and our shorter term average of D+4 from above. We now have a fairly high degree of confidence that the actual preference of the American people is around D+4. Which makes sense, considering that Democrats in congress have a higher approval rating that Republicans in congress according to every poll conducted this year.