Lewandowsky’s Unreported Results

Some information from sources at the University of Western Australia. On October 21, 2010, the following email was sent to the UWA staff mailing list:

UWA researcher Charles Hanich is seeking participants for a web-based survey of attitudes towards climate science (and other sciences) and skepticism. The survey carries no risks for participants. To participate in the survey please use this link:

Completion should take less than 10 minutes and all data will be analyzed anonymously and without monitoring or identifying individual responses.

Ref: RA/4/1/4007

[Notice approved by:
Human Research Ethics Committee,
Research Services, University of Western Australia ]

For some strange reason, the invitation is online at a web agggregator here. (I googled the survey id.)

Obvious questions:
What was the results of UWA staff who actually took the survey. Surely this would have made an interesting comparison group with the bloggers who are the target of the Moon-landing paper.

It would have been a logical comparison. Was it done and discarded? If so, why? If it wasn’t, why wasn’t it done?

42 Comments

The date shown is 21st October 2010. Can we rely on this from answerpot.com as the date Charles Hanich made the request known? And to whom? How do we know that the mailing list this was sent to was UWA only Steve?

Look at the notices, they are all related to studies, and most have UWA in the title. The one labeled “Invitation to take part in web-based survey of attitudes towards science” is authored by Charles, or as indicated in the announcement, Charles Hanich.

seeking participants for a web-based survey of attitudes towards climate science (and other sciences) and skepticism

Funny, when I initially read the above I had to go back and double-check to make sure that the invite received by Pielke Jr. was the same (it is). Something I hadn’t noticed the first time, is that the above wording suggests that one would expect the survey to be exploring participants’ attitudes towards climate science, other sciences and skepticism. i.e. How do I feel about climate science? How do I feel about other sciences? How do I feel about skepticism?

And for all we know – considering that the date is even later than the infamous Sept/2010 invites – perhaps this one was such an exploration (rather than the bait and switch it appears to be in retrospect) and the questions were completely different!

In the paper the fieldwork dates are given as Aug-Oct. The paper doesn’t actually say that all responses were from the climate blogs. All blogs known to have publicised the survey did so in late August. The last comment mentioning the survey is dated 1 September. By 23 September he had 1100+ responses (before or after elimination of duplicates?) according to his Monash presentation. So maybe the Aug-Oct fieldwork date indicates that UWA responses were used.

I’m suggesting that ‘UWA responses’ is best seen for now as responses arising from this UWA mailing list request, responses which may or may not have been from UWA staff. Until we know more. Which we certainly should by now, if Lewandowsky had any interest at all in reporting properly. Still, there’s always Steve McIntyre and Google to fall back on. Oh my.

The best way to learn about anything is to ask a large number of people at the University of Western Australia. This is an axiom of cognitive psychology and I’m surprised that it isn’t more widely appreciated.

Sorry I posted a near identical comment on a previous thead. I meant to post it on this thread, but did it in the wrong window by accident.

Anyway:

Is it just me, but I can not help but wonder about some things.

1. Could there be a link between asking UWA staff to do the survey, and going of the way to allow multiple responses from a single IP address (as previously reported on ClimateAudit)?

I mean wasn\’t the explanation given of why the methodology chosen allows multiple responses from a single IP address, that in circumstances like multiple staff responding from \”our lab\” (or similar words), this IP address flexibility would be neede.

2. ClimateAudit chooses the title of the \”Unreported Results\”. Are we sure they are unreported? Is it possible that some of the UWA staff responses may be included in the survey results?

Is there any statement from Lewandowksy, or other evidence, to exclude this possibility?

This is all the paper has on who took part:
Participants
Visitors to climate blogs voluntarily completed an online questionnaire between
August and October 2010 (N = 1377). Links were posted on 8 blogs (with a pro-science
science stance but with a diverse audience); a further 5 “skeptic” (or “skeptic”-leaning)
blogs were approached but none posted the link.

Visitors to climate blogs completed it, but it doesn’t say other people didn’t.
Since all responses from climate blogs would have been in by early September, the only reason I can see to extend fieldwork to October would be to pick up some more respondents. One can easily imagine one of the authors objecting that the 8 (or 6) blogs gave too narrow a source population, Hanich being sent off on a half-hearted and unsuccessful effort to pick up some from sceptic blogs, and a final effort being made by this internal method.
Another question to put to Lewandowsky.

Lewandowsky’s paper states, “Visitors to climate blogs voluntarily completed an online questionnaire between
August and October 2010 (N = 1377).” The university survey would fall into that window.

It seems unlikely that he would include those results in the paper. Indeed the paper itself only mentions the blogs.

But then again you have the email to Pielke Jr. Charles Hanich write, “This, however, will block legitimate use of the same computer, such as in our laboratory, where numerous participants use the same PCs.”

and if it was the same kwiksurvey, as listed on a ‘pro-science’ blog, would not these results be mixed in.. The papers methodology is so unlcear, all we can do is ask questions. the paper is vague, did ALL the respondents come from pro-science blogs, were these numbers added to by uwa staff, or not..
if not, what became of the uwa data, junkscience also – hosted the survey, with what looks like an invitation from Hanich) what became of that data..

A control survey amongst the general public (uninterested in the CC debate would have been useful) in all probability, both sceptic and ‘pro-science’ blogs, even on this flawed data, may have come out more sceptical about all the conspiracy theories than the general population…

just for amusement value, as it covers some of the conpiracy theories in the Lewandowsky paper…

The online poll, by film company 20th Century Fox·
1 Area 51 exists to investigate aliens (48%)·
2 9/11 was orchestrated by the US government (38%)·
3 Apollo landing was a hoax (35%)·
4 Diana and Dodi were murdered (32%)·
5 The Illuminati secret society and masons are trying to take over the world (25%)·
6 Scientologists rule Hollywood (17% )·
7 Barcodes are really intended to control people (7%)·
8 Microsoft sends messages via Wingdings (6%)·
9 US let Pearl Harbour happen (5%)·
10 The world is run by dinosaur-like reptiles (3%)

Here is the text at that link – points directly as noted to the survey – no blogs involved:

21-10-2010 02:54 AM – UWA researcher Charles Hanich is seeking participants for a web-based
survey of attitudes towards climate science (and other sciences) and skepticism. The survey carries no risks for participants. To participate in the survey please use this link:

The ethics of research involving human and animals have caused a proliferation in standards of practice. When research can cause some degree of harm (even as minor as a bit of emotional discomfort), participants must give informed consent. So when there’s no risk of harm, a statement saying so covers the bases. Don’t know about Australia, but in the US researchers using human subjects in surveys are frequently required by Institutional Review Boards (university committees to monitor research activities for compliance with the standards) to take several hours of training. See for example https://www.citiprogram.org/Default.asp?

Since Lewandowsky has stated that such research is useless if it doesn’t follow the standard procedure of recording all IP addresses, it will become clear whether or not the UWA responders were included when he releases this data.

Unfortunately, rather than addressing this question or any of the other serious issues, Lewandowsky’s latest post is more concerned with attacking Oliver Manuel and Christopher Monckton.

“Stephan Lewandowsky & Charles Hanich from The University of Western Australia are recruiting participants for an online study which explores attitudes towards science. Participants will be asked to complete a survey consisting of around 40 questions which should take less than 10 minutes to complete.

This study has been approved by The University of Western Australia ethics committee.

For full details and/or to take part in this research, please Click Here.”

Stephan Lewandowsky & Charles Hanich from The University of Western Australia are recruiting participants for an online study which explores attitudes towards science. Participants will be asked to complete a survey consisting of around 40 questions which should take less than 10 minutes to complete.

This study has been approved by The University of Western Australia ethics committee.

For full details and/or to take part in this research, please Click Here.

Lew and his colleague state that, recognition of “suspect responses”, or fake responses in their survey depends on a cognitive model which rests on

“(a) the tacit premise that no one could possibly be serious when endorsing all conspiracy theories, and it would include (b) the further deduction that anyone who does this must be faking the response. A further deduction (c) could be added that this faking was done in order to cast aspersions on people who reject (climate) science.”

Recognition of fake responses to any given survey, including Lew’s survey, does NOT depend ANY of the three points listed in Mr Lew’s cognitive model.

In extreme cases, recognition of fake responses can be made independent of the content of the answers to survey questions – it is a question of pattern recognition and does not involve reading the mind of the participants.

I know somebody — and he has several employees — who I believe does ascribe to all the theories — my memory may fail me and maybe he misses one or two — but that’s OK — because he and his staff have others with which they will be pleased to educate you. If he had replied and answered honestly would his answers be a “fake” — or is he simply an outlier?

I have met him, talked with him, corresponded with him and discussed common interests — in quite rational matters I assure you. I also find his staff are quite rational. They just have a different belief system than you and I and for some reason are quite open to these theories. I do admit that I can be become uncomfortable at times with him and his bunch — but 90% of the time I have no discomfort with the matters that are under discussion.

Steve probably knows of him but likely has never met him or dealt with him as while he lives not far from him they move in different worlds if I may put it that way. Also, his business has likely come to Steve’s attention — unknowingly I suspect.

My point is simply that sometimes what seems obvious is far from obvious.

If Steve contact me I will be pleased to pass along the name and business — just not publicly as I happen to think he is a “nice guy” despite this minor “character flaw”…

This is absolutely typical in psychology. If you are testing the preferences of a select group, your results will always be more robust with a control population. Most professors just use a class full of students and force them all to take the survey. It is a good way of preventing self selection bias in a control population. In Lewandowsky’s study, he was basically trying to compare two self-selected groups, skeptics and adherents. A key significance test is whether either or both groups are statistically distinguishable from the general population. Unfortunately, it looks like he did his due diligence…and then suppressed the results. As Steve would say, a trademark of climate science.

This helps to explain how they managed to find over a thousand responses to a survey that would have drawn very little interest due to the slanted nature of the questions.

I suspect, due to lack of response, they opened the survey to UWA staff who would have been sympathetically motivated to respond and suffer through the questions.

The other aspect that throws me, unless the survey electronically required that all questions be answered, its curious that all questions were answered as its very common to receive incomplete surveys.

The survey also appeared to requested response for age and gender which were not reported.

[…] compromising the data integrity, yesterday Steve McIntyre discovers that Lewandowsky’s assistant, Charles Hanich offered the survey to staff and faculty…, but there’s no mention of this in the peer reviewed paper, and no record of whether those […]