Category Archives: exit polls

Malacanang occupants should remember the voting percentages for P-Noy in the May 2010 elections (see Table 1 below). Maybe for 12-18 months these would be the core BSA3 support for key contentious issues. These would be the groups which will give him the benefit of the doubt and take his side for some time.

…..
We put the Porsche item in the press release since it’s the only other survey item directly about PNoy that is open to report; we just didn’t want to devote a separate press release to the Porsche matter alone. We could not have selected the Porsche item as “the cause” of anything, since there was nothing else to select.

Since all “agree-disagree” statements are vulnerable to affirmation bias, a question designer has to consciously choose what position is to be affirmed. It is our practice to choose the “common sense” or orthodox position, which in this case is that buying such an expensive car does not set a good example.

Suppose we had used the reverse statement that “there is nothing wrong with PNoy’s buying the Porsche since anyway it’s his own money”? To have done it that (unorthodox) way would have risked criticism for being too soft on PNoy, rather than neutral.

—————————————————————————

I defer to the long experience of SWS on these surveys, including how it designs questions,

But let us focus on the message, not the messenger.

Net Ratings

A positive net rating indicates that the respondents agree that it is not a good example that P-Noy has set by buying a Porsche, even with his own money.

A negative net rating could mean that it is a good example? Perhaps it would be more appropriate to interpret this in the way that Malacanang does. That this is a personal matter. That the President is entitled to some leisure time and can use his free time to unwind in the way he wants to, like driving fast cars or practice shooting at the range. That he did not use government funds. I wonder if this would be akin to playing golf by FVR and GMA.

But it is only in the ABC class where we observe this negative rating (-2). The D (+17) and E (+15) respondents have positive net ratings. Has the message from Malacanang only catered or made sense to the ABC class? Note that the D and E classes have little access to leisure time, access and opportunity.

According to the Family Income and Expenditures Survey of the National Statistics Office, Filipino families in 2009 spent a measly 0.4 percent of their income for recreation. Also, the average income of the top 1 percent of the income distribution was less than Php 1.9 million.

So it is still somewhat positive that a third of the D (33 percent) and E (31 percent) took the side of Malacanang, as did 48 percent of class ABC.

Going back to P-Noy’s voters last May 2011, it is interesting that 48 percent of class ABC supported him. This is the same percentage that sided with him in the Porsche issue; there is only a small percentage (6) that was undecided. Of course it would be too presumptuous to think that the same 48 percent who voted for him are the same 48 percent who sided with him here. But the conclusion is that this core among the ABC has not been eroded, nor added to.

This is not the case with voter percentages for the D and E classes versus those favorable to the ‘Porsche’ issue. For D, it is 44 vs. 33; and for E, 36 vs.31. Note also the significant percentages of the undecided, 18 percent for D and 24 percent for E. Malacanang should sharpen its focus of their messaging to the D and E groups, especially the ‘undecideds’, to keep at least his voters loyal to him.

This is of course a conservative way of targeting their constituency, at least in this Porsche issue or anything related to this genre. Otherwise it would be the sentiments of Juana Change that would offer the more compelling argument/s, as it seemed.

Well, back to the title, I am unsure if the correlation between the purchase of the Porsche and the decline in P-Noy’s ratings is significant. As Mahar had written and let me reiterate, ‘We put the Porsche item in the press release since it’s the only other survey item directly about PNoy that is open to report; we just didn’t want to devote a separate press release to the Porsche matter alone. We could not have selected the Porsche item as “the cause” of anything, since there was nothing else to select’.

Pulse Asia President Prof. Ronnie Holmes said Aquino got 41.4% of presidential preferences among 12,400 target respondents who voted in the May 10 polls nationwide. His closest rival, former president Joseph Estrada, only got 29.2%.

Thus, unlike in the exit poll of Social Weather Stations (SWS) where Binay and Roxas were tied at 40%, the ABS-CBN/Pulse Asia exit poll had Binay with a 5.3-percentage point lead over Roxas. (Click here for story.)

Tabunda said the 2010 exit poll of ABS-CBN-Pulse Asia had 63% completed interviews, which is lower than the 75% completion rate in the 2007 exit poll. She said many of the field interviewers encountered locked homes, primarily because a lot of voters were stuck in long queues at the polling precincts.

She said respondents were usually approached inside their homes, given a sample ballot and asked to fill up the ballot according to how they voted.

TNS Managing Director Gary de Ocampo said another challenge faced by field interviewers was the high number of respondents who refused to answer the ballots “probably because they were already tired.”

Tabunda said the exit poll had low completion rates from NCR respondents and from Class A,B,C. “We had more Class E respondents participate in the exit polls,” she said, adding that they had to balance the number of respondents to get a more statistically representative sample.

She said the new exit poll also had more willing respondents from Mindanao rather than Visayas, which is a possible bailiwick of Binay. The margin of error for the exit poll was (+)(-)2.

Pulse Asia also said 86% of Iglesia ni Cristo voters interviewed for the exit poll voted for Aquino and Roxas after the INC leadership endorsed the two. The INC vote is around 3% of the electorate, equivalent to at least 1.5 million votes.

WITH STILL 10% OF CLUSTERED PRECINCTS AWAITED AT COMELEC, THE ERROR MARGINS OF THE EXIT POLL RESULTS, WHEN TAKEN INTO ACCOUNT, INDICATE A CLOSE CONTEST DOWN TO THE WIRE. HOWEVER A BINAY WIN MAY HAVE A BIGGER SPREAD THAN A ROXAS WIN.

——————————————————————————–

Social Weather Stations, in cooperation with the School of Government

of Ateneo de Manila University, conducted a national exit poll sponsored

by the PLDT-SMART Foundation, Inc., for exclusive broadcast by the

Associated Broadcasting Company (TV5), called the “TV5-SWS EXIT

POLL.” The partial [underscoring mine] results for the national positions

of President and Vice-President, from the partial sample as of May 11,

1:00 a.m. and aired on TV5 as of 7a.m. today, are shown below. It will

be a long week/month for Jojo Binay and Mar Roxas. The exit poll puts both

Social Weather Stations, in cooperation with the School of Government of Ateneo de Manila University, conducted a national exit poll sponsored by the PLDT-SMART Foundation, Inc., for exclusive broadcast by the Associated Broadcasting Company (TV5), called the “TV5-SWS EXIT POLL.”

The TV5-SWS Exit Poll adheres to the requirements set by the Fair Election Act’s Section 5.5 on exit polls. This requires interviewers to (a) stay at least 50 meters away from polling centers, (b) wear distinctive TV5-SWS Exit Poll clothing and carry TV5-SWS Exit Poll identification cards; and (c) inform respondents that they may refuse to answer. The Act says that the results “may be announced after the closing of the polls on election day, and must clearly announce the total number of respondents, and the places where they were taken” and that the “announcement shall state that the same is unofficial and does not represent a trend.”

SWS previously conducted exit polls in the national elections of 1995, 1998, 2001 and 2004. Those previous polls were all conducted on election-day, in the homes of a random sample of respondents who had already voted earlier in the day. Conducting an exit poll in homes rather than at polling locations is valid under the Guidelines for Exit Polls and Election Forecasts of the World Association for Public Opinion Research (WAPOR), written by the WAPOR Exit Poll Committee of which I was a member, and approved by the WAPOR Council in 2006.

This year, however, the TV5-SWS Exit Poll is conducted not in homes but at the polling centers, 50 meters away. This enables the interviews to be done, as in the developed countries, throughout the day. The interview results are transmitted to the SWS office in Quezon City, and processed to produce the exit poll results for broadcast on TV5 as soon as completed, and lawfully allowed.

SWS conducted the TV5-SWS Exit Poll with its own regular staff and a special complement of direct hires for added interviewing and data processing capacity. Starting in mid-2004, SWS has accomplished all field and data processing operations internally, without outsourcing.

The questionnaire for the TV5-SWS Exit Poll was designed in cooperation with the Ateneo School of Government (ASoG). In addition to asking about the votes for President, Vice-President and Senator, it has several question items, in addition to standard demographics (gender, age, education, socioeconomic class, religion, charismatic group, and home language) to enable substantive analysis of the 2010 vote. The TV5-SWS Exit Poll analysis will be done by SWS in cooperation with ASoG.

SWS is entirely responsible for the sampling of the TV5-SWS Exit Poll. The poll was conducted in 802 Voting Centers (VCs) nationwide, covering all provinces from Batanes to Tawi-Tawi, and chartered cities of the Philippines. The regional distribution of the TV5-SWS Exit Poll sample of VCs, after proportional allocation according to the official 2010 number of Registered Voters (RVs) in the said provinces and cities, is as follows:

Within each province/city, the sample-VCs are chosen with probability proportional to their number of RVs. In each VC, SWS fielded normally two interviewers (sometimes one, if the VC is small); the total field group, including support staff, is over 1,800 persons. SWS has 110 encoding stations at its main office to accommodate the incoming interview reports.

The interviewers of the TV5-SWS Exit Poll have been trained to behave in a strictly non-partisan manner at all times. They are to approach voters to be interviewed at a steady pace throughout voting hours. They may not accept voters who volunteer to be interviewed. For the sake of privacy, interviewees’ names are not recorded; interviewees are given a thank-you card with the name of their interviewer. The TV5-SWS Exit Poll aims for an average of 40 interviews per VC, or a national sample of about 32,000 interviews. The national sampling error, though still to be ascertained, is expected to be below 1%.

The results of each VC are weighted by its present number of RVs, multiplied by the actual voting turnout percentage in the last presidential election year (2004), to progressively obtain provincial/city results, regional results, and national results. However, the voting results for national positions may be reported at the national level only, at which the sample is maximized; it is not necessary for an exit poll to reach conclusions about sub-jurisdictions.

The reputable Social Weather Stations released an analysis of the effect of its survey results on the voting plans of the electorate for the 2007 Senatorial elections and came out with this conclusion:

‘In the last Senatorial elections, surveys affected the plans of very few voters, and moved these very few slightly towards underdog candidates, according to analysis of the SWS May 2-4, 2007 pre-election survey.’

Thus the effects are minimal. But is this really so?

A closer look at the breakdown by socio-economic class indicates the influence to be most significant for the ABC group.

_____________________________________

VOTING EFFECTS OF ELECTION SURVEY NEWS

In percent

RP

ABC

D

E

Aware of election survey news

48

77

49

39

no effect on vote

32

55

33

26

some effect on vote

16

22

16

13

Unaware of election survey news

52

23

51

61

_______________________________________

In coming up with the summaries or obtaining averages, the ABC class has less than 10 percent weight. The D class has over 60 percent and the E class about 30 percent and thus both classes dictate heavily on the results. However it should be noted that while their numbers are limited, those in the ABC class possess most of the nation’s wealth and earn most of the income generated in the economy.

In the 2003 Family Income and Expenditures Survey conducted by the National Statistics Office, the top 1 percent (about 165 thousand families) of the income distribution earned more income (PhP 235 billion) than nearly a third of the lowest income earning families combined (about 5.3 million families) who earned some PhP 227 billion. I hope you agree with me that this is a graphic picture of the lopsided income distribution in the Philippines. It is also not difficult to imagine the immense amount of socio-economic-political power at the hands of the richest.

The SWS statistics are correct but its analysis did not sufficiently point out that the effect of its results on the electorate mattered most where it should be, – on the ABC class. Pity that the survey weights were based on the distribution of population (as is the standard practice) and not on income. I am certain that the overall results would have been different.

Note that the the final count depends on the number of votes, but the campaign to get these votes depends to a large part on the resources that a candidate can muster and have at hand to carry out the campaign strategy. These include, among others, the capability to cover the country wherever necessary, to support campaigners’ needs, to touch base with people and parties of clout and influence, to produce and place effective campaign ads in the tri-media to get awareness and commitment, to get the right profiles and sufficient exposure in the media, and to protect votes cast during the canvassing.

It would be safe to assume that those in (and with) power scan the results and based on these, tend to choose whom to support and give assistance and contribute to the cause of the ‘winnable’ candidates.

Another group affected by the results are the candidates’ supporters, – the army of volunteers (“pure” and those working below market rates). I remember that when our Senatorial candidate moved out of the magic circle of 12 and landed 15th in the survey a week before the elections, many of them were disheartened. Needless to say, their energy diminished and the morale of the ranks could be felt at headquarters.

Let me also pursue the issue of total survey error, and not only sampling error, in exit polls and the possibility of its effect on the results.

Sampling Error is the calculated statistical imprecision due to interviewing a random sample instead of the entire population. The margin of error provides an estimate of how much the results of the sample may differ due to chance when compared to what would have been found if the entire population was interviewed.

…Coverage Error…is the error associated with the inability to contact portions of the population. For example, telephone surveys usually exclude people who do not have landline telephones in their household, the homeless, and institutionalized populations. This error includes people who are not home at the time of attempted contact because they are on vacation, …, along with a variety of other reasons that they are unreachable-for the period the interviewing (with call backs) takes place.

Measurement Error is error or bias that occurs when surveys do not survey what they intended to measure. This type of error results from flaws in the instrument, question wording, question order, interviewer error, timing, question response options, etc. This is perhaps the most common and most problematic collection of errors faced by the polling industry.

Non-response Error results from not being able to interview people who would be eligible to take the survey. For example in a telephone survey, many households now have answering machines and caller ID that prevent easy contact; other people simply do not want to respond to calls sometimes because the endless stream of telemarketing appeals make them wary of answering. Non-response bias is the difference in responses of those people who complete the survey vs. those who refuse to for any reason. While the error itself cannot be calculated, response rates can be calculated and there are countless ways to do so. The American Association for Public Opinion Research (AAPOR web site) provides recommended procedures for calculating response rates along with helpful tools and related definitions to assist interested researchers.

In the SWS Exit Poll of 10 May 2004, it was able to call the eventual results within its 2 percent sampling error. (See http://www.sws.org.ph/pr051904.htm). Below is its comparison with the congressional canvass and the partial NAMFREL (National Movement for Free Elections, a ‘non-partisan’ citizen quick count) figures.

________________________________________

Comparative Results of Counted Votes for President (in %)

Choices

Congress

SWS

NAMFREL

Macapagal-Arroyo, Gloria

40.0

41

39.0

Poe, Fernando, Jr.

36.5

32

37.0

Lacson, Panfilo

10.9

9

10.8

Roco, Raul

6.5

5

7.0

Villanueva, Eduardo

6.2

5

6.2

No answer

8

_______________________________________

If the SWS results were compared with the congressional canvass, the 8 percent ‘no-answers’ in the results of its exit polls imply that SWS underestimated the votes for Poe (4.5 percent), Lacson (1.9 percent), Roco (1.5 percent) and Villanueva (1.2 percent) and overestimated that for Arroyo (1 percent). Probabilities had Poe in the worst luck with the survey results.

But knowing that total survey error is more than the sampling error, the amount of outright refusals in its exit polls is disconcerting to this writer. From the distribution of survey respondents below and from the quoted SWS release, the refusals amounted to 20 percent.

So this particular exit poll had a sampling error of 2 percent, and a non-response error of 19.7 percent. There was a targetted set of 10,620 respondents; of these only 5963 interviews were conducted. The coverage error would then be the 4657 respondents, almost 44 percent, who were not available for interview from 3-6 p.m on May 10, after polling places closed. In Metro Manila, many did not return home after voting and preferred to go malling and avoid the heavy downpour that occurred. We can assume, though, that measurement error is negligible per the findings of experts.

A review committee organized at the initiative of SWS and composed of four members from the academe and three from the Marketing and Opinion Research of the Philippines released last October 2004 its findings from the review on the exit polls. It found that the sample selection and interview instruments (ballots followed by questionnaires) of the Exit Poll are deemed methodologically sound.In addition, an audit of the field operations indicated that the design was faithfully executed, and that encoding errors for candidate choices were minimal and not deliberate.

I will not go further into an overall quantification of error because I doubt that the above errors are additive and the aggregation may become too complex.

Pulse Asia also had an exit poll conducted for ABS-CBN for the 14 May 2007 Senatorial elections. However similar information as provided above are not available, perhaps because this is the property of the broadcast corporation. Personal information on the response rates indicate a 75 percent response rate. 16 percent of the respondents were not available and another 9 percent refused to be interviewed. Thus it expressed guarded reservations on the results.

From a posting on inquirer.net on 16 May 2007, “Pulse Asia said last night that there was a statistical chance that the last four candidates on the list could be replaced by other senatorial candidates.”

And together with the concern over COMELEC counts, are we now sure that the true winners of the 2004 presidential elections and the 2007 Senatorial elections really were the ones sworn into office? It is difficult for me to accept that these election surveys have little effect on the voting patterns of the electorate, both directly and indirectly. Pollsters, you know that these do, and it is no coincidence that your peak business happens on the weeks and months leading to the elections.

However the above analysis does not diminish my view of the professionalism of these outfits. This piece would not have been possible were it not for their adherence to professional ethics and their providing adequate technical information on their surveys. I acknowledge the important role of the professional pollsters, such as those in SWS and Pulse Asia, in the exercise of suffrage and do not call for a halt to these kinds of research. But we can be more discerning in reading into the results instead of being drawn in by the screaming headlines that usually accompany these releases.

I hope that results can be released more objectively and soberly; let the media put the spin on these.

I was not home when the results of the May 2004 were processed and ‘legitimized’ with the shadow of doubt whether the vox populi was actually heard. More particularly it was bothering that this cloud of uncertainty would hang over the country at an opportune time when pro-poor reform was (and remains to be) badly needed. I decided to write this on my own 3 years ago to sublimate my rage at so many alleged attempts to hide the truth from the electorate. A Japanese university asked me to expound on the paper but I had no more time and motivation to pursue this issue further from where I was. It seemed too that the major decision makers of society were contented with the ‘official’ results…official meaning it came from the government’. Final results may also be appropriate, — in that finally let us stop working on this. True results?…you are pushing your luck. I am publishing this on the web since it may have been brought back into context by the slow count of the May 2007 elections, which as of now has yet to be concluded. Read on.