Get PhotoCrank at PhotoCrank!
January 14, 2008

Why Bother To Publish It?

When looking at polls, readers should always review the sampling to determine their predictive value. After some of the strange polling in the 2004 and 2006 election, most people have become more educated on how to spot poor polling efforts, even when presented by major media outlets like the New York Times and CBS News. One would also think that the survey debacle in New Hampshire would make news organizations like CBS and the Gray Lady a little more circumspect than before.

One would be incorrect:

Republican voters have sharply altered their views of the party’s presidential candidates following the early contests in Iowa and New Hampshire, with Senator John McCain, once widely written off, now viewed more favorably than any of his major competitors, according to the latest nationwide New York Times/CBS News Poll. ...

On the Democratic side, Senator Barack Obama’s victory in Iowa has improved his standing within the party on a critical measure: his electability. The percentage of Democrats who say he would be the strongest candidate against the Republicans has more than doubled in a month, to 35 percent from 14 percent in December.

And how did the NYT and CBS partnership determine these trends? They surveyed a grand total of 504 likely Democratic primary voters nationwide. That at least might pass the smell test, although it makes for suspect predictive value. For the Republicans, they managed only 282 likely primary voters, one of the worst efforts since the Los Angeles Times couldn't find 180 registered Republicans for its polling late last month -- and predicted that Romney had overcome Huckabee's Iowa surge.

On the basis of this sample, in which every respondent accounts for better than 0.3% of the electorate, both CBS and the Paper of Record record a huge surge in support for John McCain. It's certainly possible that this surge has materialized, but it could also be that their sample inadvertently oversampled McCain supporters. Less than 300 likely Republican voters nationwide simply doesn't have any predictive value as a sample; it's worse than the LA Times' effort, which only focused on Iowa.

It appears that pollsters took no lessons from their debacle among Democrats in New Hampshire. Are they looking for even more embarrassment? Why bother to publish such an anemically-sampled poll?

UPDATE: Here's the CBS version of the poll internals. Go to the last page to see the sampling.


TrackBack URL for this entry:

Please note that unverified Disqus users will have comments held in moderation. Please visit Disqus to register and verify your account. Comments from verified users will appear immediately.