Marcel, I always try my best to give credit where it is due, and had I known, you would most certainly have been cited. I agree, knowing of your original blog post would have saved much trouble and speculation. I’ve also left a comment with The Hockey Schtick, asking him to credit you. By way of compensation, I’ve added De staat van het klimaat to the WUWT blog roll. That said, the post did generate quite a bit of discussion, always a good thing. Marcel writes of a guest post by Koutsoyiannis:

A blog post earlier this week about an EGU presentation of Eva Steirou (a researcher in the group of Demetris Koutsoyiannis) on temperature data homogenisation created some stir in the blogosphere after Watts Up With That? and Climate Audit paid attention to it. Koutsoyiannis has now written a guest blog to give some first reactions.

Writing about the complaints that this was a presentation, and not a paper yet, he says:

But we plan to produce a peer-reviewed paper (unless we have made a fatal error, which we hope not) and we keep studying the topic more thoroughly. That is why we think that we are lucky to have received all these comments from the blogs. I did not have the time to read them all, let alone to assimilate them, so I will not provide replies here. From first glance I find most of them very useful, whether they are positive or negative.

…

But of course these are scientific disagreements and it is fine if scientists disagree. Some arguments, though, fall into other categories, such as arguments from authority or ad hominem. Well, I am familiar with such arguments within scientific transactions, formal (paper reviews) or informal (in blogs), but they are always saddening and also make it necessary to refer to personal information in order to reply.

Koutsoyiannis makes an interesting point about blogs -vs- traditional peer review. Traditional peer review is a slow and arduous process, taking months, sometimes years, and in my opinion is a holdover from a much slower time pre-Internet and email. Usually less than a dozen people are involved in that process. Blog review of papers, presentations, and topics is like an insta-launch, where citizens and scientists alike spar in sometimes a gladiatorial style over broad issues as well as minutiae. Hundreds and often thousands of eyes and minds are brought to bear, often picking the carcass clean of errors until nothing is left.

For all its warts, science blogging has a purpose and a place in today’s world. I happen to think that the occasional errors like this one, where by accident, credit wasn’t given, actually work to improve things in the long run, because as we all know:

35 thoughts on “More on Koutsoyiannis and the homogenization of temperature data – plus some comments on blog review”

Your website ends up being crowd sourcing review. In many ways, it is superior to the legacy peer review system. The IPPC should treat blog posts on WUWT as having been reviewed and worthy of inclusion in their reports.

Crowd-sourcing – whether that be in data collection such as the SurfaceStations project, data analysis such as in the many cool Zooniverse (https://www.zooniverse.org/) projects, or blog critique of research – is proving to be immensely valuable in the advancement of scientific understanding because it’s democratic. That is, it opens up participation to the talents of more than the few individuals who by chance happen to be at the right place. IOW, “On the internet nobody knows you’re a dog.” Furthermore it succeeds because of merit rather than cronyism and the rewards for most participants are intellectual rather than monetary, which dampens the tendency toward corruption.

Anthony – Good for you for making things right in terms of attribution.

Crowd review of blog posts seems to be more rigorous than the legacy peer review system. IPPC should include WUWT blog posts in their reports since they are more heavily reviewed than most of the papers included in their reports.

Hi, I fully agree that it is important to the justification of science bloggin that credit is always given to the source, as a result please p[eople always include them. I know that this is difficult, tired, late at night after a full day at the office maybe, BUT it will pay dividends. It removes yet another area of critique for the “establishment” to hang on.

Anthony, thanks for the correction. The presentation of Steirou was freely available on the web, so in some way The Hockey Schtick was free to ignore my post. But then the wording changed (paper instead of presentation) and that generated a lot of noise. Anyway, it’s good that we now discuss temperature homogenisation again and I am looking forward to more constructive discussions in the future.
Marcel

Good move to correct the record on sources. Due credit when known is extremely valuable and everyone should make an effort to trace ideas back to their source.

I think we are seeing a turning point in the scientific review process. The old formal review process has demonstrated it is (and has been) broken for some time. In theory it is a good concept, but in implementation, human frailties cause it to creep toward a corrupt process unless a strong hand is at the editorial wheel that enforces good practice and no back room manipulation on the review process.

As crowd source blog review becomes more mainstream, I see us moving toward a combined process, where an initial presentation of the idea is posted, and quickly screened for errors by literally hundreds or thousands of reviewers. Each poking at their area of interest or expertise. Sometimes bringing to bear unique experience or exposure to obscure knowledge that often leads the original poster to sources and problems that would have gone unnoticed in a formal review by a small group of specialists. Then after that quick reasonableness check the formal paper is refined and presented for formal review by a broad range of specialties, not just the primary specialty of the author.

We see it often here where climate is examined from the point of view of process control engineers, electrical engineers (feedback, capacitive and inductive circuits), physics, chemistry and geology specialists just to name a few specialties that come to mind from recent discussions. In doing so they bring fresh ways of looking at the problem and sometimes expose what appears to be a poorly understood process that in another specialty (such as process control — ie feed backs ) is a very well studied and understood field.

Suddenly the original poster is opened up to areas of knowledge that he/she simply did not know existed. The depth of human knowledge today is so vast that no one person can possibly know even a small fraction of it.

This is evidenced by “climate scientists” who dabble in computer programing and statistics and think themselves competent in those areas when they are in fact blundering amateurs, making sophomoric level mistakes.

Unfortunately the formal review process by experts in their narrowly defined field only reinforces those faulty perceptions of confidence because the reviewers often do not themselves have journeymen level understanding of statistics or computer coding good practices, and proper code validation and testing methods.

In time I think we will see that the “instant review” of the blogs and crowd sourcing backed up by formal review of multiple specialists such as statistics, higher mathematics, physics etc. will help to break down the current faults of the peer (pal) review system as it exists today.

Please someone publish a paper on the frailties of computer models, and the inherent risk of undocumented bugs in all computer code, even well tested code. Model output is not data it is a method to make quick reasonableness checks on a theory or evaluate its sensitivity to initial conditions “if the model is correct”. For example aerodynamic analysis models do a good job of allowing aerodynamic designs to be developed on the computer but they are not data, that is only gathered in wind tunnel testing and flight testing to prove the model expectations were close to reality.

As the comments in the post mention, the write, review, publish, and wait (post-publish commentary) method of producing science literature has just about been destroyed. There are way to many print publications and articles of questionable value. Many are familiar with this, especially those in an academic setting that have been asked by your chair or dean about the number of your publications due out or accepted for publication before your next performance review.

This current (minor) kerfuffle was easily defused by Anthony – as others have said above. Demetris K. indicates the value of the digital (web) posting and review. The old way will continue to fade. The new way is not yet clear. Still, we move on —

Thanks to sites such as WUWT, the hosts, moderators, and commenters.
~~~~~~~~~~~~~~
Note: I just took the hint and visited Marcel Crok’s site. See Anthony’s second line in this post for the link.

~Gary says:
July 20, 2012 at 8:30 am
“IPPC should include WUWT blog posts in their reports”
Thanks for the giggle! The difference is that drafts of submitted papers get revised and improved as a function of peer review. Here, things just get repeated over and over. Not quite the same thing.

—–

Example(s), please? Did you forget the Geurgis (sp) paper, among likely many others?

I enjoy reading the papers, research and comments on WUWT. I’m an accountant, not a scientist per se, but I’ve had a lot of experience in a lot of different things to get an idea when something is not quite “right”. Many times the comments on WUWT help confirm my suspicions and fine tune my BS meter. And I am one of those who have come to have no faith in the current peer review process when it comes to “climate science” research. So thank you Anthony and all you bloggers and commenters for your work, and all the entertainment you provide.

Larry Ledwick (hotrod) says:”
I think we are seeing a turning point in the scientific review process. The old formal review process has demonstrated it is (and has been) broken for some time. In theory it is a good concept, but in implementation, human frailties cause it to creep toward a corrupt process unless a strong hand is at the editorial wheel that enforces good practice and no back room manipulation on the review process.
As crowd source blog review becomes more mainstream, I see us moving toward a combined process, where an initial presentation of the idea is posted, and quickly screened for errors by literally hundreds or thousands of reviewers. ”

The EGU journal Atmospheric Chemistry and Physics has an open review. Open in the sense that the comments of the reviewers are posted on the web and in the sense that everyone is invited to comment. In practice the number of reviewers is a little less than “literally hundreds or thousands of reviewers”, which is why the journal also always requests two official reviews. It would be great if one could get some sort of continual review process going.

I find D. Koutsoyiannis’ work and comments always cogent, interesting and very well thought through. Much like Watts’ essays and comments, except the very well thought through part is sometimes short due to the immediacy of daily blogging. (that is okay too part of the process) The way science blogging and mass review has developed over the years is both democratic and refreshing. The more formal science review process has its problems but it has its strengths too. It is my belief that we need both approaches and that when blended, in the long run, we all will benefit. Like most things in the world this blending is a work in progress and find the observing and participating in it highly stimulating.

“There is scarcely any well informed person ,who, if he has the will, has not also the power to add something essential to the general stock of knowledge, if he will only observe regularly and methodically some particular class of facts which may most excite his attention, or which his situation may best enable him to study with effect.” Herschell – quoted in The Zoologist 1860

“Koutsoyiannis makes an interesting point about blogs -vs- traditional peer review. Traditional peer review is a slow and arduous process, taking months, sometimes years, and in my opinion is a holdover from a much slower time pre-Internet and email.”

True.
Inefficiency is orders of magnitude beyond what we can afford to tolerate.

–“Blog review of papers, presentations, and topics is like an insta-launch, where citizens and scientists alike spar in sometimes a gladiatorial style over broad issues as well as minutiae. Hundreds and often thousands of eyes and minds are brought to bear, often picking the carcass clean of errors until nothing is left.

For all its warts, science blogging has a purpose and a place in today’s world.”

One possibility is that most sensible parties steer clear of volunteering blog comments (although they may be reading). At present the “blog review” system is operating like a corrupted justice system — corrupted by coupled pools of ignorance &/or deception.

One possible way around this is more competition — for example (no offense intended – only advancement of collective human lucid awareness intended) WUWT needs at least a handful of serious competitors with at least equal readership. I would suggest that some of these widely-read competing blogs should have different core foci & rules.

I will now suggest what I would personally find most efficient (efficiency is a high personal value due to severe constraints on time & resources):

3. Bad Apples:
Strictly prohibit protracted, presumptively-“authoritative” swipe-quote harassment (Peter Principle, Law of Authoritative Ignorance &/or Deception). Judith Curry has the idea with one of her blog rules, which suggests saying what YOU have to say (as opposed to aiming solely to protractedly harass others ad infinitum). Ignorant &/or deceptive “critics” can be allowed to concisely say what they have to say ONCE. After that they are unacceptably wasting everyone’s time if they continue with dark negativity & criticism rather than switching to offering positive, truly constructive, possibly even inspiring, observation-based alternatives. The guilty parties here are bankrupt of exploratory vision and should be cut off from further commentary after 3 back-&-forths (to keep it simple) if they’re incapable of independently exercising appropriate restraint. Just a few bad apples ruin the tone of the whole blog. I have 2 WUWT regulars in mind with this suggestion for new, competing blogs (but I refrain from naming them).

from which I wrote my own interpretation, extracted and transcribed slides and the abstract, and created my own post.

I don’t know where the tipster found this link and therefore don’t consider this an “error” or lack of courtesy, but nonetheless give Marcel credit for having found this link to a very interesting presentation and for the additional information provided by his 2 posts.

I am also not the source of the confusion about this paper being “peer-reviewed” as I never claimed that it was in my post. Having “presented papers” at over two dozen international scientific conferences myself, I am well aware that the purpose of meetings is to present and discuss new research findings often prior to even writing a paper and submitting it to formal peer review. Nevertheless, the scientific community still calls this “presenting a paper” even though the written version often hasn’t yet been prepared or submitted for formal peer-review.

I, of course, agree that credit for finding interesting links should be acknowledged when known.

@Larry
Is this now a difference ‘twixt open source and propriority science?

Yes if I understand your question, the open source software model is a useful point of comparison, vs the proprietary software developers where the software is only released for “beta testing” after it is fairly far along in its development cycle and often there are imbedded in the code assumptions about how to handle issues and code certain modules which lock in the developers per-conceptions of what the public want in the product.

Often we find that the developers in that model add bells and whistles that the majority of users have no interest in and will never use, and make assumptions about how the user will interact with the software that may not mirror reality.

The AGW community is doing the same thing. Imbedded in their models at very core levels are assumptions about what the climate is doing, and when the model output does not match reality they fiddle with the knobs to try to get the match they want but never consider the idea that perhaps some of their core assumptions are bogus.

It is like the pride of authorship of a writer that makes him blind to faults in his novel, or equally bad the blindness of an editor to a different style of writing than what he likes.

This is an excellent thread, once again the best of WUWT at work. I agree with pretty much everything said so far. I too think blogs are really important for the future of science.

BUT

There is a big caveat. I have seen at least two instances here where I felt the peer-reviewing process failed signally. I remember two because they left a bad taste in my mouth which has still not completely gone, probably because the core issues have not yet been sufficiently addressed or sufficiently compensated elsewhere.

The first failure was the review of the Sky Dragons book. What caused the failure was the disproportionate focus on the contribution of one individual who is known here as holding a rather strange singularity of interest with the sun. Not only was this contribution removed; the problem was compounded by material that came to light about this individual. In the whole process, valuable material in the rest of the book was simply drowned. In the sense of book review, Amazon manages far better IMHO.

The second failure was the handling of Ned Nikolov and Karl Zeller. I know there were faults on all sides. But the upshot is almost certainly that WUWT lost a lot of competent readers at that point. Still, I always try to find the positive in everything, and the upshot for me was, in the end, good, no, more than that, astounding…

…I went back to the material to check, check and check again. I obsessed with the science of Nikolov and Zeller for a long time, finally got my head round their maths, found it really worked as they said, found their assessment was correct, that the IPCC S-B equations are wrong… so next I started a wiki for Climate Science where I might get their material in a form that not only read far more easily but also was open to updating and correction… but then ran into the problem of their apparent contravention of the Second Law of Thermodynamics… so I dived into that… read up the lives of Kelvin and Maxwell, travelled up to Scotland to come as close to their spirit and legacy as I could… then I dived into the utterly amazing work of Graeff who has verified experimentally what Loschmidt said 150 years ago, challenging Clerk Maxwell’s interpretation of the Second Law… and upholding Nikolov and Zeller… I travelled to Germany to meet Graeff because I wanted to get to the bottom of his experiments and claims… omg it all holds up, this is Nobel-quality material but the universities won’t take on replication because… because… because Nobody Challenges The Second Law of Thermocynamics (except perp.mob.nutcases)… well actually Prof Sheehan has been running conferences on 2LoT and he’s interested in Graeff… so now the wiki is on hold (asleep not dead) while I “gird up my loins” to do this replication myself (with anyone else interested) that the universities should be doing but are not. Well, not so far… heck who am I to be making these claims, retired, no science degree… but it’s needed and it promises to correct something so fundamental that Climate Science has been wrong-footed right from the start…

I fully agree, the traditional peer-review model is obsolete and should be augmented by open access blog review.

Many of the most important discoveries in science are made when there is cross-fertilisation of ideas from different disciplines. But that becomes harder as knowledge grows exponentially and subjects of study become subdivided, specialized and compartmentalized.

Climate scientists go to conferences to meet other climate scientists. People from related disciplines will attend, of course, but experts in unrelated subjects will not.

If scientists want make use of the Internet but still restrict review of their work to other scientists then surely they could do this on forums where the membership is restricted and everyone’s qualifications are known. However, I think they would be wrong to (only) do this.

Everybody has some expert knowledge in something. Sometimes an insignificant piece of information from one area of expertise can be highly significant in another. Science blogs are important not only because they provide a forum for scientists but because they enable anyone and everyone to contribute. There is no way of knowing where the next insight will come from and innocent questions are often the most insightful.

I’ll give two examples from my own experience. They are only to illustrate a point. I don’t think either is particularly significant or likely to lead to someone’s eureka moment but I have no way of knowing.

1. Some have suggested mitigating global warming by modifying the albedo of urban environments e.g. using white (or lighter coloured) roofing materials. Many mocked this idea but I think it has merit. It reduces urban heating and the building’s requirement for air conditioning and (in some situations) heating.

2. The urban heat island effect is well known. Some warmists choose to dismiss it. Agricultural land use also changes the albedo of the earth but not always by increasing it, as some scientists seem to suggest. I knew this long before I had heard the terms UHI or climate change as I used to hang glide. Any hang glider (pilot) will tell you that a ploughed field is a good place to catch a thermal. If you want to see a small heat island in action just look at where the birds are circling.http://tinyurl.com/cxn245j

I also argue for increased albedo roofing. There are good reasons why in many hot sunny locations, houses are painted white.

I live in a hot sunny location (Perth) and pretty much all new roofs are of reflective metal that look white from above, and which keep the house cooler in summer.

Although the downside is that it makes houses colder in winter. It occurs to me that someone who invents a roofing material that changes from low to high albedo at around 25C or by the flick of a switch, will get very rich, and save more energy than all the world’s renewable energy projects put together.

Anthony, thank you very much for this and the previous post. I was really glad to see some misunderstandings being resolved here, as well as the comments by Marcel Crock, Hockey Schtick and Victor Venema, who all contributed with their own blogs in this discussion. I thank all three–and of course all commenters (and will acknowledge them if the paper gets published).

Victor, almost all EGU journals (not only Atmospheric Chemistry and Physics) have an open and public review process (see a list in http://www.egu.eu/publications/open-access-journals.html and follow the link “Papers in Open Discussion” in each one). In my view this is a great progress and I hope other professional societies, like AGU, will follow the example of EGU. But as you correctly point out, in practice the number of reviewers/commenters is small–as yet.

The blogs attract comments more easily, because of the more informal style. I had some experiences where papers were in open review in an EGU journal and at the same time discussed at blogs. While the blog discussers were aware that they could post a comment at the official journal site, they did not. This I understand very well and I would say I like it, because it is more democratic and dismantles the idea that a journal is a unique space where all experts and authorities are pent.

In conclusion, I believe that a synergistic interaction of blogs and journals provides the best service for the scientific progress.

Demetris Koutsoyiannis says:
July 21, 2012 at 3:55 am
“Anthony, thank you very much for this and the previous post. I was really glad to see some misunderstandings being resolved here, as well as the comments by Marcel Crock, Hockey Schtick and Victor Venema, who all contributed with their own blogs in this discussion.”

I saw on Victor’s blog some rant about the WUWT posts and skeptics overall, also justifying homogenisation with the need to correct for UHI.
This is an alibi for the homogenisation as it is done.
We all know the UHI effect to be real and introducing a significant difference in the temperature between the urban and natural environment, on the other side the adjustments for UHI are so minuscule or non-existent that it is a misdirection to talk about it.