Gay Marriage: Is PPP Data Skewed Right?

No, I don't mean skewed right in the traditional statistical sense where most of the data falls on the right side of the mean.  I am wondering as to whether Public Policy Polling surveys give a greater weight to Republican voters than is warranted by national party ID numbers.  Today, the PPP released numbers for a national survey, identifying 33% of voters in favor of same-sex marriage, and 57% opposed to it.  Those numbers are drastically different from a CNN Opinion Research Poll two days ago, which showed 52% of voters think that marriage should be available to gay couples, and 46% who disagreed.

What could possibly explain this, other than CNN perhaps being totally wrong, and support for gay marriage actually still at a low level while a strong majority opposes it?  I thought I'd look at the party identification numbers obtained by PPP for this survey and compare them to the national overall party ID.  Here's what I found.
Democrat: 39%
Republican: 35%
Independent/Other: 25%
It looked to me like the Republican population was overrepresented here. Turns out, I'm right. Polster.com's aggregate party ID data has these numbers:
Democrat: 32.1%
Republican: 24.7%
Independent: 35.9%
As you can see, both Republicans and Democrats are overrepresented in the PPP poll, while independents are underrepresented.  To be specific, Democrats are overrepresented by about 7 points, Republicans by more than 10 points, and independents are underrepresented by more than 10 points.  Now, if you assume that independents are roughly split same way as the two parties or this election year around are leaning more Republican than in the past, then perhaps it won't make any difference.

But polling data shows that on social issues, independents are considerably more progressive than Republicans.  In fact, even PPP's poll shows that.  On the question of legalizing same-sex marriages, Democrats are split 47-40 in favor, Republicans 81-12 opposed, and independents also opposed, but only by a 48-41 margin.  Much closer to the Democratic numbers than Republican.  So what happens if you adjust the PPP data to fit the aggregate party ID from Polster.com, while keeping each individual party response intact?  We'd still get only 33% in support, but only 50% opposed to same sex marriage.

Take a look at the PPP break-down one more time.  It seems that they not only have a party ID skewing, but also a conservative skewing within the party ID's.  Even a Fox News poll finds 56% of Democrats in favor of marriage equality, and it pegs Republican support at 37%.

Now, PPP has a reputation for accuracy in predicting election results - especially in swing states - likely because their formula figures out who is likely to turn out and they will vote well in swing states where the party gaps are not big.  But this is an issue poll, and a measure of public sentiment rather than a vote in any specific state.  And in that, it does seem like PPP has a conservative slant.

Update: Nate Silver has more on the discrepancies - question wording (constitutional right vs legal/illegal), sample frames (CNN surveyed adults whereas PPP registered voters, but 93% of CNN's surveyed were registered voters too), CNN having a bigger sample (although not by much), and automated (PPP) robocall polling vs live operator (CNN).  Nate also discusses the peculiarities in the data in addition to party ID intra-party ideology questions.


Like what you read? Chip in, keep us going.