Inside the Represent.Us poll

Given the news coverage and a handful of queries I’ve received about a recent poll commissioned by United Republic’s Represent.Us, and the recent interest in their work by noted election attorney Bob Bauer, I asked Public Campaign Action Fund’s Political Director, Jeff Robinson, to take a look at the polling memo released by the organization and the underlying top lines. His response is below. -- David Donnelly, Executive Director

I have reviewed the recent polling from Represent.Us and find this research to be somewhat inconsistent in its findings and analysis, and falls short of standard scientific polling methodology. Moreover, the analysis presented in the research firm's memo doesn't reflect the data from the top lines. While we see polls released carrying an agenda all the time, this particular analysis argues for a change in direction in both messaging and policy, which is not supported by their own data.

They argue that 1) corruption should replace a broader money in politics narrative about solutions, and 2) a policy of a "Tax Refund/Voucher" out performs small donor matching funds or a grant based system as a preferred policy. Their top lines show an insignificant finding on the first and a misleading finding on the second.

On methodology: The standard accepted methodology for public opinion research among voters is based on a scientific random sample among either registered voters or registered likely voters. This is done either by a randomization of phone numbers or as best practices indicates, a random sample of the voter file using both land lines and cell phones. This panel of voters is a recruited online via incentives with no verification from either a voter file selection or voter history available from a voter file. Nate Silver (Five Thirty Eight) critiqued online surveys by writing, “The central challenge that Internet polls face is in collecting a random sample, which is the sine qua non of a scientific survey. There is no centralized database of e-mail addresses, nor any other method to ping someone at random to invite them to participate in an online poll. Many people have several e-mail addresses, while about 20 percent of Americans still do not go online at all.” Even if we accept the methodology, there's still some misreading of the data on messaging and policy.

On messaging: The polling memo argues that this research shows a paradigm shift in the messaging around campaign finance reform towards using the term "corruption" or "anti-corruption." There are some interesting findings here, but overall, the breathlessness of the case they make for shifting language is not supported by what are modest differences in data. There's an uptick in intensity when using corruption instead of campaign finance, but overall support goes from low 90s to mid-to high 90s. That's just not that important as a strategic matter.

In the survey's Question #13, they present a split sample of the following question; "Is it important or unimportant that our elected leaders reduce the influence of A: money B: corruption in political elections?" Both of these messages test in the 90 percentile with one testing at 92% and the other at 97% with a 3.5% margin of error. They test the same basic assumptions in Question #14 as well "campaign finance laws vs. anti-corruption laws" with both of these messages testing in the 90 percentile. Once voters support your underlying message narrative at the 90% level a few percentage difference has no strategic significance, yet in the memo asserts that this is an important messaging paradigm shift. It's an interesting finding but not meaningful in any way that informs strategy.

On policy: Much more troubling and at the center of the polling analysis memo is the assertion that one policy – the tax refund one – is significantly stronger than another. This again is not supported by the data. The polling memo argues that the "Tax Refund/Voucher" model outperforms other models.

Question 15 on tax refund/grants for contributors: 69/70% support Question 16 on public grant to candidates: 77% support Question 17 on small donor matching funds: 74% support

Their finding that more people chose refunds over the others when asked to chose between the three is both misleading and irrelevant because the leading policies in Congress over the past few years have blended these proposals, not separated them.

It should also be noted that in in the final panel of questions where they test the "Anti-Corruption Act," they are mixing the testing of messages and provisions of the Act. The messages are generic and not all specific to the proposed Act, like "reduce the influence of money" these are not true tests of the proposal’s provisions, as we might do on a ballot measure, to test the strength or weakness of the proposal. These generic message tests are a stretch to use in driving strategy of even to test voters’ attitudes about the proposed Act itself.

Anyway, it is too bad that this poll wasn't fielded in a way to inform the campaign finance reform community and its supporters. An accurate interpretation of the data is that there is strong support for reform whether it's called anti-corruption or campaign finance reform, and also deep support for all the varieties of public funding the poll tested. What we could have learned, if this survey was designed to inform the community, was a deeper understanding and fine-tuning of the messages and policies, this survey did not accomplish this.