Articles and Analysis

Mickey Kaus has been arguing over the last week that the greater emphasis on racial issues in the Clinton-Obama nomination contest may have caused a return of the so-called Bradley-Wilder effect. The term refers to a pattern observed in the 1980s and early 1990s when the white opponents of African American candidates would do better on election day than indicated in pre-election polls (explained in more detail here, links to other sources here). Kaus thinks we may have evidence of a re-emergence of Bradley-Wilder in the results from the Gallup Daily and the Rasmussen Reports automated tracking surveys. Last Sunday, he wrote:

Gallup's national tracking poll has Obama retaking the lead over Hillary after bottoming out on the day of his big race speech. Rasmussen's robo-poll, on the other hand, shows Obama losing ground since last Tuesday. True, even Rasmussen doesn't seem to be putting a lot of emphasis on his survey's 6-point shift. But isn't this week's primary race exactly the sort of environment--i.e.., the issue of race is in the air--when robo-polling is supposed to have an advantage over the conventional human telephone polling used by Gallup? Voters wary of looking like bigots to a live operator--'and why didn't you like Obama's plea for mutual for understanding that all the editorial pages liked?'--might lie about their opinions, a phenomenon known as the Bradley Effect. But they might be more willing to tell the truth to a machine. ...

On Tuesday, I noted that the overall results from Gallup and Rasmussen were not that different when looking at data collected from March 14 (the day the Wright story broke) through the previous day:

Live Interviewer Gallup Daily: Clinton +2 (47% to 45)

Automated Rasmussen Reports: Obama +1 (45% to 44%)

Kaus updated his original post and responded that we should be focusing on the trends:

[I]f you look at the trend since Obama's 3/18 speech--which is what arguably charged the campaign with high-minded condemnation of racism and MSM sympathy for Obama of the sort that might produce a Bradley Effect--Obama gains 6 points in Gallup and loses 6 in Rasmussen through last Friday (and he's since lost one more on Rasmussen). That seems like a non-small difference. ...

He has continued to note the difference:

[3/25] Obama has now lost a net of 8 points on Rasmussen since the 18th, and 11 points since the 14th. On Gallup, he's gained several points.

[Yesterday] Bradley still in the race: Gallup (telephone poll) and Rasmussen (robo-poll) continue to diverge.

I have been puzzling over the trend and thought it would be helpful to post a chart of the data in question. In the chart below (click for a larger pop-up version), Kaus is focusing on the trends since the Obama/Wright speech on March 18. The dates on the chart are end-dates for each survey release. Keep in mind that Gallup reports a three-day rolling average, and Rasmussen reports a rolling four-day average, so the trend line reactions would theoretically lag slightly behind events.

If you overlook today's release, the chart does show a largely divergent trend, though most of the difference occurs in the three to four days after the speech. However, if you step back and look at the complete time series, the Gallup and Rasmussen lines are no more divergent now than they have been all along. In fact, if you remove three days of live-interviewer Gallup data -- the March 16-18 release which had Clinton (the white candidate) leading by seven points -- the divergent trend largely disappears.

Yes, for the last week, Obama has done better in the Rasmussen (automated) survey than the Gallup (live interviewer) data, but the difference is on the same scale as similar gaps since January that have see-sawed back and forth, favoring neither candidate consistently. So call me crazy, but I just don't see compelling evidence of a return of the Bradley-Wilder effect in these data, especially keeping in mind the potential for random variation in the trends.

One interesting pattern here -- and to be honest, I'm not sure what to make of it -- involves large gaps opposite of what we would expect from the Bradley-Wilder effect throughout much of January and again briefly centered on February 5/6 and March 18: At those times, the white candidate (Clinton) does better on the live-interviewer (Gallup) surveys than on the automated (Rasmussen) surveys. Also, as the charts below demonstrate, most of that difference occurs in the percentage supporting Clinton, not the percentage supporting Obama.

The difference in January may have had something to do with how the two surveys asked about other candidates still in the race, and the gaps afterward purely random, although that's a pure guess. Anyone have any better theories?

Comments

the_real_truth:

Mark-

How about this - the "Bradleyette" effect. Males (and perhaps some females) are more likely to tell a robo caller that they don't support Clinton compared to a live (female) surveyor. Or perhaps their wife is listening to the survey and so they are inclined to say they support Clinton - you know - for a little "action" later that night.

Mark B. wrote: "Yes, for the last week, Obama has done better in the Rasmussen (automated) survey than the Gallup (live interviewer) data"

You mean the opposite - Obama has done better on Gallup than on Rasmussen in the past week.
The vote share for either candidate also shows it - post-March 18, Senator Obama has done better on Gallup than on Rasmussen, while Senator Clinton has about the same support on either poll.

Still, the wait-and-see approach is best - let's see what the polls say in a couple of weeks.

Maybe in January, a Bradley-effect-for-Women was in operation? ;-) [I guess the_real_truth beat me to it!]

Looking at these two curves together (which is very nice), I had a few questions:

1) Are these significantly correlated? (chi-by-eye says yes, but that can be misleading.)
2) Are they significantly correlated on a particular time scale? This is harder to tell, especially with the different time averaging. You might be able to compute coherence (or to get really fancy, wavelet coherence), although maybe you don't have enough samples.

This might be a way of detecting some real trends. That is, from an assumption of underlying mean values plus noise (as made in earlier posts), one would assume that departures about the mean values would be uncorrelated (apart from the short time series and differences in time averaging). But maybe there is a time interval (say, a week or two) over which the two time series are correlated enough that there is something useful here.

P.S. I know Kaus sort of helped you get this whole web thing going, but it's best not to pay too much attention to him. He routinely latches on to one or two day bumps in tracking polls, and generally appears to have a real problem with statistics (or really with numbers of any sort).

This whole debate is kind of silly. What evidence is there in recent years of the so-called Bradley-Wilder effect anyway? None, as far as I can tell. Look at the 2006 Tennessee Senate race, for example. The polls were generally accurate, if anything they slightly underestimated Ford's support at the end. And we've seen pre-primary polls this year that have underestimated and overestimated Obama's support with no clear pattern as far as I can tell.

And then there is the fact that it is Obama who has generally outperformed the polls (measured the way you do on this site through the late aggregate), not Clinton (as Kauss suggest would be the case under 'Bradley.')

I don't know why no one has pointed out the obvious in accounting for the different results in Rasmussen versus Gallup. Sure, the robo polling versus human factor might have something to do with the variation, but I'm surprised that no one mentioned the voter screen that each uses.

Specifically, Gallup tallies all registered voters and apparently has not much of a screen. Rasmussen tallies only likely voters, so obviously eliminates some participants based on whatever criteria they have decided constitutes a likely voter. I don't think we need to look for subtle explanations such as robo vs. human when we have very simple divergences in method.

Also, Rasmussen's poll is a 4-day tracking poll; Gallup is 3-day. This explains why sometimes Rasmussen is slower in picking up trends than Gallup.

I hope that someone can shed light on this subject. I believe that ironically, there was no Bradley effect in the Bradley or Wilder race. The standard definition of the Bradley effect is that a portion of voters will mislead pollsters about supporting candidate x in order to give a more PC response or not to appear racially prejudiced. Thus, the non-white candidate will significantly underperform his polling results relative to election results.

What about the actual data?
Here is the last Field poll in 1982 for Tom Bradley vs. George Deukmejian

I've been thinking this one over, trying to find any other explanation. For example, Rasmussen if anything over-estimated Obama in Texas and Ohio. That goes against the "Bradley Effect Neutral" argument for IVR.

I would like to ask this:

What are the target demographics for each of these polls?

To make this much of it, you have to assume that IVR vs. human is the only difference between them. I'll bet it's not, and there is another key demographic difference that explains the (slight) gap between them.

Doesn't it rest on the assumptions that (1) these voters feel ashamed to admit they won't vote for Obama because they feel morally obligated to vote for the black candidate, and (2) care enough about what the anonymous caller thinks of them that they lie in order to make themselves look better? And since when do we assume a significant number of voters lie to pollsters, period?

Is there any basis for these assumptions? Is there any research that shows that some Dems feel they "should" vote for Obama because he's black, and feel so guilty about not supporting him that they are ashamed to admit it?

This is a Democratic primary. There are reasons for Dems to vote for either candidate. We're not talking lifelong Dems who for the first time are planning to vote Republican because the Dem nominee is black.

If there is a trend of overstated support for Obama compared to actual votes tallied, the obvious reason is the one that has been suggested here before, and again by the_real_truth and RS above. One would lie about supporting Obama because supporting Hillary has become highly stigmatized. If you think about the way each candidates supporters have been "explained" in the popular press, you've got the energized youth and the inspired African Americans on the one side, vs. the ditzy women who support Hillary because she cried, the sexist women who support Hillary just because she's a woman, the ignorant uneducated whites, southern racists and old people.

Even the Bradley effect meme is just another way of keeping the message out there that Clinton supporters as closet racists.

First of all, I'm not convinced by the "Bradley Effect" one way or the other. I keep wondering if we'll have a chance to measure it very directly in this election or rule it out forever, but I haven't seen a clean experiment yet.

So as an agnostic, I want to make it clear that this isn't about "racism", and I really hope no one takes it that way. This is one of those subtle grey areas that is more about social behavior generally.

People *think* they are supposed to vote for a minority or woman candidate, and that this is the appropriate thing to do - but have their own reasons why they won't in the end. What's important is that they are worried about their image to a stranger - and a racist wouldn't necessarily be.

Once again, it's not been proven that any of this is real, but I keep hoping one way or the other. But please, don't think of it as calling someone "racist" - it's far more complicated than that.

I'd like to suggest a little different take on the subject. If there is a Bradley/Wilder effect, and I'm not convinced there is, it may be that looking for it in expressions of voter preference may be less fruitful than in "favorable/unfavorable" ratings generated from IVR vs live interviews.

As noted on another thread, I'm struck by the much greater variation in polling organizations' fav/unfav ratings than in voter choice. IVR polls (SurveyUSA and Rasmussen) appear (at least by according to my impression) to generate much lower favorability and much higher unfavorable ratings than polls with live interviewers.

Since the effect seems to impact all three candidates, I'm not sure it's a "Bradley/Wilder" effect, but my first impression is that the differences are greater for Obama than for the other two candidates.

In short, fav/unfav ratings may be a more sensitive measure than the various dichotomous voter choice items.

Post a comment

Name:

Email Address:

URL:

Comments: (you may use HTML tags for style)

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.