HolyCoast: Before You Trust The Poll, Check the Samples

Wednesday, March 14, 2012

Before You Trust The Poll, Check the Samples

Yesterday Pew Research had a poll out showing Obama with a 50% approval average, a definite outlier compared to several other polls showing Obama in the low 40's.  How'd that happen?  The polling sample had 51% Democrats.

Obama's approval polls are sure to go up and down in the coming months, but before you trust any particular poll, be sure to check the samples used because manipulating the number of Democrats to Republicans can make bad results look a lot better (from Morning Jolt):
I would just tell everyone to be very mindful of the partisan breakdown in these samples. Ed Morrissey assessed the Washington Post sample over atHot Air:

After hiding the partisan composition numbers for the last two months, today's poll includes that data from the last three presidential polls. Let's take a look at the D/R/I series, keeping in mind that the D/R/I from the 2010 midterm elections was 35/35/30: 

March: 31/27/36

February: 34/23/37

January: 32/25/36

The February poll from which the WaPo/ABC series derived its previous Obama approval numbers gave Democrats an 11-point edge and undersampled Republicans by 12 points in relation to their 2010 turnout. This survey has a difference of seven points in the gap.

Now let's take a look at the sample in the most recent CBS News/New York Times poll: 26 percent Republican, 30 percent Democrat, 35 percent independent. 

The sample in their preceding poll in February: 27 percent Republican, 34 percent Democrat, 32 percent independent. A shift from D+7 to D+4.

If the Democrats have a big numeric advantage in your sample, Obama will do better, and as it gets more even, he will do worse. Remember, the partisan breakdown in 2008 was Democrats by 7, and that's pretty much the best-case scenario for the party (exhaustion and dissatisfaction with GOP incumbent, unpopular war, economic collapse, historic nominee, glowing media coverage, some Republicans with mixed feelings about their nominee).

I mention this because we may see "the Obama comeback" in the next round of polls. I'm telling you now, check the samples. 
There will be all kinds of games played with these numbers, depending on what Obama needs to either look good or look better.

No comments: