Yesterday, I noticed that the Washington Post had published another of its series of polls, and I decided to take a look through the sampling. Given that the only use of it last night was for analyzing the Republican primary race, in which only Republican responses got used for data, I decided to hold off on writing about it until the Post used the overall data for other purposes. This morning, Peter Baker obliged with a story about George Bush’s historic low approval ratings:
President Bush is a competitive guy. But this is one contest he would rather lose. With 18 months left in office, he is in the running for most unpopular president in the history of modern polling.
The latest Washington Post-ABC News survey shows that 65 percent of Americans disapprove of Bush’s job performance, matching his all-time low. In polls conducted by The Post or Gallup going back to 1938, only once has a president exceeded that level of public animosity — and that was Richard M. Nixon, who hit 66 percent four days before he resigned.
I’m not going to argue that Bush doesn’t have low approval ratings or that he is solidly unpopular at the moment. I think that has been pretty well established, especially after he alienated his own base over immigration reform — if you’ll pardon the pun. However, the Post’s polling has such a sampling problem that it calls into serious question how accurately they could measure his disapproval ratings.
After a few years of relative equality, Democrats have pulled ahead of Republicans in party affiliation, as NBC noted in February. Nationally, Democrats enjoy a 34.3%-30.4% advantage in registrants. This has caused some analysts to predict that the GOP will have a tougher time in the Electoral College than in the last two elections, which was the general point of the article.
Now let’s look at the Washington Post sample. On question 901, respondents answered that they were 35% Democrats, which is close enough to the national average. However, only 23% identified themselves as Republicans, which amounts to a 24% underrepresentation (see update below) of the GOP in this sample. In fact, the Post consistently underrepresents Republicans, and has for the past two years. The last time it came close to reality was in November 2006 — when the Post needed to make sure its election predictions came close to the results.
Not surprisingly, that was also the last time the Post’s polling on George Bush’s approval ratings came close to reality, too. His disapproval then was 57%, which the elections seem to have confirmed. At the time, Rasmussen — which has been historically more accurate than the Post — had it at 56%. They now have it at 59%, actually down from a high of 65% in the first part of July during the immigration debate.
Bush is not popular, by any means. However, by seriously underrepresenting Republicans in its polling samples, the Post exaggerates his unpopularity and renders its polling unreliable. If their pollsters cannot generate a sample that resembles the American electorate, then they should find new pollsters.
UPDATE: A few people have questioned the “24% underrepresentation” remark, but it’s accurate. If the proper representation of Republicans in the electorate is 30.3%, and the Post’s sample is only 23% Republican, then it underrepresents Republicans by 24% — not because I think its off by 24 percentage points and it should be 47%, but because it only includes 76% of the proper Republican sample. That means the sample of Republicans is off by 24%. That’s not insignificant, and therefore any general conclusions made from this poll are unreliable at best.
Welcome readers of RealClearPolitics and The Guardian!