Search Forums

If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Re: Big Win for Climate Scientist

Breaking story coming from a well respected scientist at NOAA who claims that NOAA used unreliable data that was later discarded in order to rush this study out for the Paris talks. He notes that NOAA violated its own review procedures and exaggerated the model in order to remove the pause.

A high-level whistleblower has told this newspaper that America’s National Oceanic and Atmospheric Administration (NOAA) breached its own rules on scientific integrity when it published the sensational but flawed report, aimed at making the maximum possible impact on world leaders including Barack Obama and David Cameron at the UN climate conference in Paris in 2015.

The report claimed that the ‘pause’ or ‘slowdown’ in global warming in the period since 1998 – revealed by UN scientists in 2013 – never existed, and that world temperatures had been rising faster than scientists expected. Launched by NOAA with a public relations fanfare, it was splashed across the world’s media, and cited repeatedly by politicians and policy makers.

But the whistleblower, Dr John Bates, a top NOAA scientist with an impeccable reputation, has shown The Mail on Sunday irrefutable evidence that the paper was based on misleading, ‘unverified’ data.

Re: Big Win for Climate Scientist

The ever-reliable science denial vector – Daily Mail has come out with another set of accusations about NOAA temperatures.
I guess they didn’t realize that those findings have already been confirmed in a well vetted Journal.

Scientific American:

Various studies have debunked the idea of a pause, or hiatus, in global warming—the contention that global surface temperatures stopped rising during the first decade of this century. The arguments for and against “the pause” were somewhat muted until June 2015, when scientists at the National Oceanic and Atmospheric Administration published a paper in Science saying that it had slightly revised the sea surface temperatures it had been citing for the 1900s. The measurement methods, based on sensors in the engine intake ports of ships, had been flawed, NOAA said. The revised methodology also meant that sea surface temperatures during the 2000s had been slightly higher than reported. NOAA adjusted both records, which led to a conclusion that global surface temperatures during the 2000s were indeed higher than they had been in previous decades. No hiatus.

Critics attacked NOAA, claiming it had cooked the books to dismiss claims of a pause. Republican Rep. Lamar Smith of Texas opened a congressional investigation of NOAA scientists, including demands that they turn over their emails, which they have not.

Now independent scientists have weighed in. A study published Wednesday in Science Advances shows that the adjustments NOAA made were justified. A team led by Zeke Hausfather at the University of California at Berkeley and Kevin Cowtan at the University of York analyzed raw data from buoys, satellites and robotic sensors around the world’s oceans. They concluded that the old methods had indeed overestimated sea surface temperatures in the past—but that the newer calculations had underestimated temperatures for the 2000s.

This kerfuffle is part of a larger effort by the now-greatly-empowered fossil fuel industry to crush science in favor of “alternative facts”. The attack on this specific NOAA temperature data has been going on for a while, and was the topic of my video of last year:"

"In an article in today’s Mail on Sunday, David Rose makes the extraordinary claim that “world leaders were duped into investing billions over manipulated global warming data”, accusing the US National Oceanic and Atmospheric Administration (NOAA) of manipulating the data to show more warming in a 2015 study by Tom Karl and coauthors.

What he fails to mention is that the new NOAA results have been validated by independent data from satellites, buoys and Argo floats and that many other independent groups, including Berkeley Earth and the UK’s Met Office Hadley Centre, get effectively the same results.

NOAA’s results are independently verified
The new NOAA record published in Karl et al primarily updated their ocean temperature record. While they also released a revised land record based on data from the International Surface Temperature Initiative (and the related Global Historical Climatology Network version 4 beta product – GHCNv4), the land record was largely similar to their prior record and was responsible for relatively little of the increase in warming they showed.

I recently led a team of researchers that evaluated NOAA’s updates to their ocean temperature record. In a paper published last month in the journal Science Advances, we compared the old NOAA record and the new NOAA record to independent instrumentally homogenous records created from buoys, satellite radiometers, and Argo floats. Our results, as you can see in the chart below, show that the new NOAA record agrees quite well with all of these, while the old NOAA record shows much less warming.

This was due to two factors: the old NOAA record spliced together warmer ship data with colder buoy data without accounting for the offset between the two; and the new NOAA record puts more weight on higher-quality buoy records and less weight on ship records (versus the old NOAA record which treated ships and buoys equally). You can read more about the study in Carbon Brief’s article.

The fact that the new NOAA record is effectively identical with records constructed only from higher quality instruments (buoys, satellite radiometers, and Argo floats) strongly suggests that NOAA got it right and that we have been underestimating ocean warming in recent years.

John Kennedy, a researcher at the UK’s Met Office in charge of their ocean temperature product, agrees that NOAA’s new record is probably the most accurate in the last two decades, remarking: “At a global scale, those adjustments really do seem to work and the ERSSTv4 adjustments [NOAA’s new record] work best of all.”

Rose’s claim that NOAA’s results “can never be verified” is patently incorrect, as we just published a paper independently verifying the most important part of NOAA’s results.

NOAA’s land data similar to other records
The land record that NOAA used in the Karl et al paper was a sneak peak at their new GHCNv4, which increases the number of global land stations from the 4,400 currently used to around 25,000.

This is quite similar to the Berkeley Earth land temperature record, which uses a similar number of stations. There is little reason to think that the inclusion of more station data will give us less accurate results.

The Karl et al land record ends up quite similar to the old one, though it shows about 5% more warming since 1970, mostly attributable to the inclusion of additional stations in the Arctic. As the chart below demonstrates, their results are quite similar to those of Berkeley Earth as well as the current operational NOAA land record (based on GHCN version 3), and also agree quite well with the latest version of GHCNv4.

The new NOAA temperature record is also by no means an outlier when compared to other groups producing global (land and ocean) surface temperature records. It shows less warming in recent years than records from Berkeley Earth, NASA, and Cowtan and Way, and a bit more warming than found in the Hadley Centre/CRU record. The old NOAA record, on the other hand, was on the bottom of the pack, with less warming than found by the other groups.

If folks don’t like the NOAA data, they will get the exact same story using surface temperature data from any other group, with no detectable sign of a “hiatus” or “pause” through to the present.

NOAA did make their data available
In his article, David Rose relies on reports from a researcher at NOAA who was unhappy about the data archiving associated with the Karl et al paper. While I cannot speak to how well the authors followed internal protocols, they did release their temperature anomalies, spatially gridded data land and ocean data, and the land station data associated with their analysis. They put all of this up on NOAA’s FTP site in early June 2015, at the time that the Karl et al paper was published.

As someone who works on and develops surface temperature records, the data they provided would be sufficient for me to examine their analysis in detail and see how it compared to other groups. In fact, I used the data they provided shortly after the paper was published to do just that. While it would have been nice for them to publish their full analysis code online as well as the data, I’m sure they would have provided it to any researchers who asked.

Rose also makes a big deal about the fact that NOAA’s new ocean temperature product adjusts buoys up to match ship data versus adjusting ship data down to match buoys. This turns out to be a bit of a red herring; since scientists are interested in the change in temperatures over time, you end up with the same increase in temperatures (e.g, the temperature trend) if you apply the offset to one or the other.

Because climate scientists work with temperature anomalies (rather than absolutes), the direction of the offset doesn’t have any effect on the resulting temperature series. On the other hand, not correcting for the offset between ships and buoys results in a spurious cooling bias, and a record that differs a lot from the buoys themselves as we found in our paper.

Rose’s article presents a deeply misleading graph where he shows an arbitrary offset between NOAA’s data and the Hadley land/ocean dataset. This is an artefact of the use of different baselines; Hadley’s “0C” value is relative to the average temperature from 1961-1990, while NOAA’s is relative to the average temperature from 1901-2000 (a period which includes the colder early 20th century).

This comparison ends up being spurious, because each record uses a different baseline period to define their temperature anomaly. As the chart below shows, when you correctly put the two datasets on the same baseline (eg, with respect to the 1961-1990 period), you find no offset in recent years between the two, though there is slightly more warming in the NOAA dataset due to the higher weight they give more reliable buoy data in their analysis.

Similarly, if you simply download the NOAA and Hadley ocean temperature datasets you would find that the published Hadley values are actually higher than the published NOAA ocean values in recent years. This is because Hadley uses a 1961-1990 baseline for their ocean temperature product while NOAA uses a 1971-2000 baseline. Putting both datasets on a common baseline is essential to performing accurate comparisons.

Further updates to come from NOAA
NOAA is planning on further updates to their sea surface temperature record this year to incorporate Argo data and to make some adjustments to their spatial interpolation technique. Based on the preliminary results that their team presented at the American Geophysical Union meeting late last year, their new record (ERSSTv5) will have about 10% less warming than their current record (ERSSTv4) over the 2000-2015 period, largely due to changes in the way that they account for areas with limited data. Their upcoming record will still show 50% more warming than the old NOAA record (ERSSTv3b).
While NOAA’s data management procedures may well need improvement, their results have been independently validated and agree with separate global temperature records created by other groups.

The “astonishing evidence” that David Rose purports to reveal in no way changes our understanding of modern warming or our best estimates of recent rates of warming. It does not in any way change the evidence that policymakers have at their disposal when deciding how to address the threats posed by climate change.

If anything, there is strong independent evidence that NOAA’s new record may be the most accurate one over the last two decades, at least for the two-thirds of the world covered in ocean."

Re: Big Win for Climate Scientist

Originally Posted by CowboyX

"In an article in today’s Mail on Sunday, David Rose makes the extraordinary claim that “world leaders were duped into investing billions over manipulated global warming data”, accusing the US National Oceanic and Atmospheric Administration (NOAA) of manipulating the data to show more warming in a 2015 study by Tom Karl and coauthors.

Interesting, though clarifying point. David Rose made no such claim. He reported a claim by Dr John Bates a scientist with an impeccable reputation.

Originally Posted by Cowboy

The new NOAA record published in Karl et al primarily updated their ocean temperature record. While they also released a revised land record based on data from the International Surface Temperature Initiative (and the related Global Historical Climatology Network version 4 beta product – GHCNv4), the land record was largely similar to their prior record and was responsible for relatively little of the increase in warming they showed.

This is a perfect example of a Red Herring fallacy.

Notice that he is talking about the underlying data set being valid. The problem with that defense is that Dr. Bates never questioned the underlying data set. He had two objections.

1) That the smoothing method (pairwise homogeneity adjustment) was unproven and remains so.

2) That the internal review procedures were bypassed and internal objections to the paper were suppressed by NOAA management.

Neither of those are related to the actual measurements by observational tools. IMO, that kind of defense is carefully calculated to appeal to those who have little background in modeling. They generally don't know of the myriad steps between raw data and projection and so assume that if the underlying data is correct, everything is fine.

The problem with the underlying data is that it shows virtually no warming since the 1970s. But it shouldn't because there are all kinds of problems with that underlying data that need to be clarified, NOAA has done a lot of good work in that field. NOAAs smoothing based on the urban heat island effect, for example, is more or less widely accepted.

The complaint here is about identifying outliers. Sometimes tools record data incorrectly. A warm water spot may hit the sensor, or a wave might cause it to accidentally read air temp, or a thousand other technical errors. NOAA is (correctly) attempting to spot and remove those outliers from the data set to create a more reliable data set.

The problem is, the method they are using has not been validated either internally or independently. You'll notice in the second article the author validates the dataset, but only references that they released GCHNv4, not that they validated it. His defense of GCHNv4 is likewise a bit of a Red Herring, he argues it is valid because it isn't as extreme as other data sets, which is a ridiculous defense for a modeler. The fact that the geocentric model is less extreme than the stationary spheres model is hardly a valid defense.

This had undergone the critical process known as ‘pairwise homogeneity adjustment’, a method of spotting ‘rogue’ readings from individual weather stations by comparing them with others nearby.

However, this process requires extensive, careful checking which was only just beginning, so that the data was not ready for operational use. Now, more than two years after the Pausebuster paper was submitted to Science, the new version of GHCN is still undergoing testing.

Moreover, the GHCN software was afflicted by serious bugs. They caused it to become so ‘unstable’ that every time the raw temperature readings were run through the computer, it gave different results. The new, bug-free version of GHCN has still not been approved and issued. It is, Dr Bates said, ‘significantly different’ from that used by Mr Karl and his co-authors.

Dr Bates revealed that the failure to archive and make available fully documented data not only violated NOAA rules, but also those set down by Science. Before he retired last year, he continued to raise the issue internally. Then came the final bombshell. Dr Bates said: ‘I learned that the computer used to process the software had suffered a complete failure.’

The reason for the failure is unknown, but it means the Pausebuster paper can never be replicated or verified by other scientists.

And that is the most telling point. If NOAA hasn't approved of or validated GCHNv4, why should we accept it?

Originally Posted by Cowboy

Rose’s article presents a deeply misleading graph...

This is a very valid criticism of the article. I believe we've had a discussion along this lines before, but on reverse sides. The graph definitely appears to be from the Mail rather than the Dr. Bates. There are myriad problems with it, there is actually a great discussion of those problems in the comments section of the link above.

Zeke has written a Factcheck on David Rose’s article. His arguments are that:
1NOAA’s sea surface temperatures have been independently verified (by his paper)
2NOAA’s land surface temperatures are similar to other data sets
3NOAA did make the data available at the time of publication of K15

With regards to #1: In a tweet on Sunday, Zeke states

>Zeke Hausfather ‏‪@hausfath
‪@KK_Nidhogg‪ @ClimateWeave @curryja and v5 is ~10% lower than v4. Both are way above v3, which is rather the point.

What Zeke is referring to is a new paper by Huang et al. that was submitted to J. Climate last November, describing ERSSTv5. That is, a new version that fixes a lot of the problems in ERSSTv4, including using ships to adjusting the buoys. I managed to download a copy of the new paper before it was taken off the internet. Zeke states that v4 trend is ~10% lower than v5 for the period 2000-2015. The exact number from information in the paper is 12.7% lower. The bottom line is that sea surface temperature data sets are a moving target. Yes, it is good to see the data sets being improved with time. The key issue that I have is reflected in this important paper A call for new approaches to quantifying biases in observations of sea surface temperature, which was discussed in this previous CE post.

Regarding #2. Roger Pielke Sr. makes the point that ALL of the other data sets use NOAA’s GHCN data set. Zeke makes the point that CRUT and Berkeley Earth do not use the homogenized GHCN data. However, as pointed out by John Bates, there are serious problems with the GHCN beyond the homogenization J

Regarding #3. John Bates’ blog post states: “NOTE: placing a non-machine readable copy of a dataset on an FTP site does not constitute archiving a dataset”

Victor Venema has a blog post David Rose’s alternative reality. The blog post starts out with a very unprofessional smear on the Daily Mail. He provides some plots, cites recent articles by Zeke and Peter Thorne. Nothing worth responding to, but I include it for completeness. The key issues of concern are in John Bates’ blog post (not what David Rose wrote).

The fundamental issue is this: data standards that are ‘ok’ for curiosity driven research are NOT ‘ok’ for high impact research of relevance to a regulatory environment.

Also hilariously, one of the co-authors of the original K15 paper also argues that Mr. Hausfather is playing fast and loose with his interpretation and defense.

>Thomas Peterson ‏‪@tomcarlpeterson
16h Buoys read 0.12C cooler than ships. Add 0.12C to buoys or subtract 0.12C from ships and you’ll get exactly the same trend.

Response: Well, in the new Huang et al. paper on ERSSTv5, it turns out that adjusting the ships to buoys results in a trend that is lower by 0.07oC. NOT exactly the same – in the climate trend game, a few hundredths of a degree actually matters.

IBID

Getting back to the defense of GHCNv4 though, it isn't actually clear, and K15 authors refuse to clarify, which version was used, the discarded v3 or the more updated, but unconfirmed v4. One of the criticisms of the K15 team was that they refused to archive the data sets they used. While the raw data is available as your link points out, the processed data was not saved as required by NOAA procedures. Procedures established by Dr. Bates following a paper and national award on the subject by Dr. Curry. https://judithcurry.com/2017/02/04/c...-climate-data/

It is clear that the actual nearly-operational release of GHCN-Mv4 beta is significantly different from the version GHCNM3.X used in K15. Since the version GHCNM3.X never went through any ORR, the resulting dataset was also never archived, and it is virtually impossible to replicate the result in K15.

...

The lack of archival of the GHCN-M V3.X and the global merged product is also in violation of Science policy on making data available [link]. This policy states: “Climate data. Data should be archived in the NOAA climate repository or other public databases”. Did Karl et al. disclose to Science Magazine that they would not be following the NOAA archive policy, would not archive the data, and would only provide access to a non-machine readable version only on an FTP server?

Results that cannot be replicated in science have the same academic weight as opinions. If I can't verify your results, they aren't valid.

The question about which data reconciliation model was used is critically important because it seems somewhat likely that v3 was used. But that version was disavowed by NOAA (before the paper's publication date btw which makes it academic fraud not just a bad decision). https://www.ncdc.noaa.gov/sotc/natio...emental/page-5

We also need to be clear that this isn't some unassailed paper that only "deniers" question. There was widespread academic criticism of the paper when it was published. Multiple peer reviewed articles were published highlighting data errors, modeling errors, odd decisions (why did they use the lax 90% standard rather than the virtually universal 95% or 99% confidence level). [Fun experiment, take a class in statistics and try to submit a report with a 90% confidence level, you'll be failed by your professor].

Peer Reviewed paper criticizing the data retention and handling methods of the K15 paper and showing that the model construction methodology violates standard procedure. The authors actually recommend a full retraction. http://journals.ametsoc.org/doi/10.1...S-D-15-00015.1

A great direct response to the paper's errors in a letter to the editor (the science version of a rebuttal):

Key pitfalls of the paper:
◾The authors have produced adjustments that are at odds with other all other surface temperature datasets, as well as those compiled via satellite.
◾They do not include any data from the Argo array that is the world’s best coherent data set on ocean temperatures.
◾ Adjustments are largely to sea surface temperatures (SST) and appear to align ship measurements of SST with night marine air temperature (NMAT) estimates, which have their own data bias problems.
◾The extend of the largest SST adjustment made over the hiatus period, supposedly to reflect a continuing change in ship observations (from buckets to engine intake thermometers) is not justified by any evidence as to the magnitude of the appropriate adjustment, which appears to be far smaller.

1. They make 11 changes (not all are explained) producing the ERSSTv4 Sea Surface Temperature (SST) dataset that includes new estimates for the different way SSTs are measured from ships (intake or buckets). They also add 0.12°C to each buoy to bring their measurements in line with those taken from ships. These issues have been raised before by the UK Met Office when compiling their HadSST3 ocean surface temperature dataset, see, ‘A review of uncertainty in in situ measurements and data sets of sea surface temperature’

2. The greatest changes are made since 1998, which is interesting because this is when we have the highest quality of data and global coverage using several methods. Only this analysis finds any increase in global annual average surface temperature over this “hiatus” period. The authors have produced a dataset that is at odds with other surface temperature datasets, as well as those compiled via satellite.

3. The authors start their trend estimates in 1998 and 2000. This has long been considered unwise as 1998 is a very strong El Nino year and 1999-2000 is a much cooler La Nina period. The difference between them distorts their trend estimates. For example, their 1998-2014 trend is 0.106+/- 0.058°C per decade. Starting two years later (during La Nina influenced years) yields a trend of 0.116 +/- 0.067°C per decade as one would expect from starting at a lower temperature. Ignoring these caveats the authors say their analysis produces twice as much warming for 1998-2014 than earlier estimates. Their conclusion is, ironically, based on inbuilt biases in their analysis.

Their Fig 1 shows that when using their updates it is only with the use of these inappropriate start and end points that the “hiatus” is reduced.

4. Even with the 11 changes to their SST database and the problem of start and end dates the authors admit that the statistical significance of their results is only significant at the 0.10 level, and in some cases not even that.

“I believe their estimates of the error in their decadal trend figures are far too small. They quote the error in a 15-year period to a precision of one thousandth of a degree C. In their report the authors admit that their error analysis is not definitive and that looking at them another way invalidates their trend conclusions,” said Dr David Whitehouse, science editor of the GWPF.

5. Note that trends that include 2014 and 2015 must be treated with caution due to a recently persistent very warm feature in the NE Pacific that is affecting global SST estimates.

6. In addition, they do not include any data from the Argo array that is our best coherent data set on ocean temperatures. The authors state this is because Argo temperature data is not surface data. However, ship-derived temperatures can be from as much as 15 m below the surface. The Argo array samples 5 m below the top of the ocean. From 2004 to 2013 it shows considerable variation and little trend. The non-ARGO data aptly demonstrates the problem of starting trend analysis in 1998 or 2000.

8. Extending a change in ship observations (from buckets to engine intake thermometers) to the present time had the largest impact on the SST adjustments over the hiatus period, per Karl et al 2015:

“Second, there was a large change in ship observations (i.e., from buckets to engine intake thermometers) that peaked immediately prior to World War II. The previous version of ERSST assumed that no ship corrections were necessary after this time, but recently improved metadata (18) reveal that some ships continued to take bucket observations even up to the present day. Therefore, one of the improvements to ERSST version 4 is extending the ship-bias correction to the present, based on information derived from comparisons with night marine air temperatures. Of the 11 improvements in ERSST version 4 (13), the continuation of the ship correction had the largest impact on trends for the 2000-2014 time period, accounting for 0.030°C of the 0.064°C trend difference with version 3b.”

Ref (18) is a 2011 paper by Kennedy et al. It states (paragraph 3.1) “Dating the switchover from uninsulated canvas buckets to insulated rubber buckets is problematic as it is not clear how quickly the practice of using insulated buckets was adopted. … Based on the literature reviewed here, the start of the general transition is likely to have occurred between 1954 and 1957 and the end between 1970 and 1980.”

A 2010 review article “Effects of instrumentation changes on SST measured in situ” by Kent, Kennedy, Berry and Smith states that “Models of corrections for wooden and uninsulated canvas buckets show the adjustments to be five to six times greater for the canvas buckets.”

So post 1980 adjustments to bucket measurements should be very small (under 0.1 C) Moreover, by 2000 ship measurements were a minority of total measurements and all types of bucket were a small proportion of ship measurements (see figs 2 and 3 of Kent et al. 2010). These facts imply that post 2000 adjustments warranted by use in some ships of bucket measurements should be negligible.

“The justification given for the change that had the largest impact on trends for the 2000-2014 time period – continuing to adjust ship SST measurements by reference to night marine air temperature (NMAT) data, ‘which have their own particular pervasive systematic errors’ (Kennedy 2014) – i.e. that some ships still continue to take bucket observations, appears to support only a very small adjustment,” said Nic Lewis, an independent climate scientist.

In summary
This is a highly speculative and slight paper that produces a statistically marginal result by cherry-picking time intervals, resulting in a global temperature graph that is at odds with those produced by the UK Met Office and NASA.

Caution and suitable caveats should be used in using this paper as evidence that the global annual average surface temperature “hiatus” of the past 18 years has been explained.

It should be noted that Hadley is recommending that you not use the data set that NOAA used for this paper.

An interesting rebuttal to the K15 paper in that its models do not conform to post model observations. I find this personally compelling because that is a basic requirement for model validation. In all real model building a basic requirement is to segment off a portion of your data to not be included and used later as verification data (ie does the model accurately predict that test data). For some reason this is often discarded in climate science (really the only field I know of that does this, financial, genetic, population dynamic, forensic, and a host of other fields all require this for peer review). This work takes data collected by the same sources and runs it through the K15 model. The variance is significant enough that it violates statistical significance and as such tells us the model is not accurately predicting observational results. http://surfacetemperatures.blogspot.....html?spref=tw

Peer Reviewed critique of the joining method defended in the article you cited. Significant because it argues there is no way to conduct this kind of outlier search objectively and so, by definition, includes observer bias in the result. http://rda.ucar.edu/datasets/ds277.0...14-00006.1.pdf

Peer Reviewed paper critiquing the models put forward in K15. Author uses the gold standard ARGOS data and finds that the K15 conclusion to be spurious if a more reliable data set is included. This is an important note. The K15 paper puts a lot of weight on engine intake readings from ships. The problem is that these are tools used primarily for engineering, not scientific collection and suffer from conductive exaggeration (the metal is hotter than the surrounding water. If we match these to the most reliable data set we have ARGOS we see a pretty wide divergence. K15 chose to use the bouys as the metric, but the bouys have a shorter operational lifespan and data set meaning the comparison is inherently less accurate. http://rda.ucar.edu/datasets/ds277.0...14-00007.1.pdf

"Suffering lies not with inequality, but with dependence." -Voltaire

"Fallacies do not cease to be fallacies because they become fashions.” -G.K. Chesterton

Re: Big Win for Climate Scientist

Originally Posted by Squatch347

Peer Reviewed paper critiquing the models put forward in K15. Author uses the gold standard ARGOS data and finds that the K15 conclusion to be spurious if a more reliable data set is included. This is an important note. The K15 paper puts a lot of weight on engine intake readings from ships. The problem is that these are tools used primarily for engineering, not scientific collection and suffer from conductive exaggeration (the metal is hotter than the surrounding water. If we match these to the most reliable data set we have ARGOS we see a pretty wide divergence. K15 chose to use the bouys as the metric, but the bouys have a shorter operational lifespan and data set meaning the comparison is inherently less accurate. http://rda.ucar.edu/datasets/ds277.0...14-00007.1.pdf

Yes, but:

"Based on the preliminary results that their team presented at the American Geophysical Union meeting late last year, their new record (ERSSTv5) will have about 10% less warming than their current record (ERSSTv4) over the 2000-2015 period, largely due to changes in the way that they account for areas with limited data. Their upcoming record will still show 50% more warming than the old NOAA record (ERSSTv3b)."

Re: Big Win for Climate Scientist

Originally Posted by CowboyX

Yes, but:

And? Neither of these data sets include the ARGOS data, which is considered, even by your reference, as the gold standard in sea temperature. This response seems to have little to do with the objections I put forward. It doesn't deal with:

1) The data refinement methodology used.

2) The violation of approved data retention and review procedures.

3) The absence in NOAA data of the widely accepted gold standard within comparison sets.

What you are implicitly assuming is that one poorly managed data set is better than an earlier poorly managed data set. Not only is that potentially not the case given the reliance on non-scientific measuring tools, but it is absolutely irrelevant to the arguments laid out against K15.

"Suffering lies not with inequality, but with dependence." -Voltaire

"Fallacies do not cease to be fallacies because they become fashions.” -G.K. Chesterton

Re: Big Win for Climate Scientist

Originally Posted by Squatch347

And? Neither of these data sets include the ARGOS data, which is considered, even by your reference, as the gold standard in sea temperature. This response seems to have little to do with the objections I put forward. It doesn't deal with:

1) The data refinement methodology used.

2) The violation of approved data retention and review procedures.

3) The absence in NOAA data of the widely accepted gold standard within comparison sets.

What you are implicitly assuming is that one poorly managed data set is better than an earlier poorly managed data set. Not only is that potentially not the case given the reliance on non-scientific measuring tools, but it is absolutely irrelevant to the arguments laid out against K15.

"NOAA is planning on further updates to their sea surface temperature record this year to incorporate Argo data and to make some adjustments to their spatial interpolation technique." (post 22 source)

"In his article, David Rose relies on reports from a researcher at NOAA who was unhappy about the data archiving associated with the Karl et al paper. While I cannot speak to how well the authors followed internal protocols, they did release their temperature anomalies, spatially gridded data land and ocean data, and the land station data associated with their analysis. They put all of this up on NOAA’s FTP site in early June 2015, at the time that the Karl et al paper was published.

As someone who works on and develops surface temperature records, the data they provided would be sufficient for me to examine their analysis in detail and see how it compared to other groups. In fact, I used the data they provided shortly after the paper was published to do just that. While it would have been nice for them to publish their full analysis code online as well as the data, I’m sure they would have provided it to any researchers who asked." (same source)

Re: Big Win for Climate Scientist

"NOAA is planning on further updates to their sea surface temperature record this year to incorporate Argo data and to make some adjustments to their spatial interpolation technique." (post 22 source)

That seems to be a widely popular decision by NOAA, and would seem to be the right thing to do. Here is the problem related to this discussion. It has no relevance to the paper or criticism. K15 didn't use that ARGOS data, right?

The relevant criticism by Dr. Bates is:

1) The data refinement methodology used.

2) The violation of approved data retention and review procedures.

3) The absence in NOAA data of the widely accepted gold standard within comparison sets.

Originally Posted by Cowboy

"In his article, David Rose relies on reports from a researcher at NOAA who was unhappy about the data archiving associated with the Karl et al paper.

Just again to be clear, the criticism was leveled by Dr. Bates, not just some researcher, but a senior researcher in charge of developing and monitoring the process for creating research papers and archiving data.

Second, this claim was already rebutted by Dr. Curry.

Regarding #3. John Bates’ blog post states: “NOTE: placing a non-machine readable copy of a dataset on an FTP site does not constitute archiving a dataset”

Victor Venema has a blog post David Rose’s alternative reality. The blog post starts out with a very unprofessional smear on the Daily Mail. He provides some plots, cites recent articles by Zeke and Peter Thorne. Nothing worth responding to, but I include it for completeness. The key issues of concern are in John Bates’ blog post (not what David Rose wrote).

The fundamental issue is this: data standards that are ‘ok’ for curiosity driven research are NOT ‘ok’ for high impact research of relevance to a regulatory environment.

Cited earlier.

and

Getting back to the defense of GHCNv4 though, it isn't actually clear, and K15 authors refuse to clarify, which version was used, the discarded v3 or the more updated, but unconfirmed v4. One of the criticisms of the K15 team was that they refused to archive the data sets they used. While the raw data is available as your link points out, the processed data was not saved as required by NOAA procedures. Procedures established by Dr. Bates following a paper and national award on the subject by Dr. Curry. https://judithcurry.com/2017/02/04/c...-climate-data/

It is clear that the actual nearly-operational release of GHCN-Mv4 beta is significantly different from the version GHCNM3.X used in K15. Since the version GHCNM3.X never went through any ORR, the resulting dataset was also never archived, and it is virtually impossible to replicate the result in K15.

...

The lack of archival of the GHCN-M V3.X and the global merged product is also in violation of Science policy on making data available [link]. This policy states: “Climate data. Data should be archived in the NOAA climate repository or other public databases”. Did Karl et al. disclose to Science Magazine that they would not be following the NOAA archive policy, would not archive the data, and would only provide access to a non-machine readable version only on an FTP server?

Your author essentially is making the argument that he believes, in theory that he could reconstruct the data sets via the raw data. That is incorrect for three reasons.

a) That isn't how research publication works. If you create a modified data set (as K15 did) you are supposed to publish that modification online, both by NOAA rules and more general peer review rules.

b) it is unsupported since he hasn't actually done so.

c) It isn't the case, several researchers have tried and failed to get the same results.

From my earlier post:

Peer Reviewed paper criticizing the data retention and handling methods of the K15 paper and showing that the model construction methodology violates standard procedure. The authors actually recommend a full retraction. http://journals.ametsoc.org/doi/10.1...S-D-15-00015.1

...

◾The authors have produced adjustments that are at odds with other all other surface temperature datasets, as well as those compiled via satellite.
◾ Adjustments are largely to sea surface temperatures (SST) and appear to align ship measurements of SST with night marine air temperature (NMAT) estimates, which have their own data bias problems.
◾The extend of the largest SST adjustment made over the hiatus period, supposedly to reflect a continuing change in ship observations (from buckets to engine intake thermometers) is not justified by any evidence as to the magnitude of the appropriate adjustment, which appears to be far smaller.

1. They make 11 changes (not all are explained) producing the ERSSTv4 Sea Surface Temperature (SST) dataset that includes new estimates for the different way SSTs are measured from ships (intake or buckets). They also add 0.12°C to each buoy to bring their measurements in line with those taken from ships. These issues have been raised before by the UK Met Office when compiling their HadSST3 ocean surface temperature dataset, see, ‘A review of uncertainty in in situ measurements and data sets of sea surface temperature’

2. The greatest changes are made since 1998, which is interesting because this is when we have the highest quality of data and global coverage using several methods. Only this analysis finds any increase in global annual average surface temperature over this “hiatus” period. The authors have produced a dataset that is at odds with other surface temperature datasets, as well as those compiled via satellite.

In summary
This is a highly speculative and slight paper that produces a statistically marginal result by cherry-picking time intervals, resulting in a global temperature graph that is at odds with those produced by the UK Met Office and NASA.

Caution and suitable caveats should be used in using this paper as evidence that the global annual average surface temperature “hiatus” of the past 18 years has been explained.

...

Peer Reviewed critique of the joining method defended in the article you cited. Significant because it argues there is no way to conduct this kind of outlier search objectively and so, by definition, includes observer bias in the result. http://rda.ucar.edu/datasets/ds277.0...14-00006.1.pdf

Re: Big Win for Climate Scientist

Originally Posted by CowboyX

You're welcome to start a thread and continue debating this, I'm not interested since they've moved to v5 with similar results.

I don't think the conclusion that it produces similar results is warranted. You are talking about them moving to a new raw data set, not that the new data set has been run through the methodology in K15.

There has been no ERSSTv5 data presented related to "the pause." This is due primarily to the fact that NOAA hasn't certified ERSSTv5 as I noted in my initial rebuttal. It might be the latest, greatest, but even its owner NOAA hasn't approved it for use in scientific documents. Nor have independent researchers verified the methodology or data set because it is still under development by NOAA.

Originally Posted by Cowboy

I'll tentatively agree with all of the criticism you've made.

Ok, I think there is a debate going on in the scientific community, but the bulk of the argument seems to be how much this paper is incorrect rather than whether or not it accurately reflects warming in the 1995-2010 period.

The larger problem with this paper is that it was published in violation of the existing controls and that makes a lot of very reputable scientists nervous as to the procedure for NOAA data and publication. You'll notice a lot of discussion in the peer-reviewed papers recommending that NASA data be given higher weighting in future modeling over NOAA sea data based on (a little bit) this methodology issue and (more so) the failure to validate their reconciliation models (GCHNs).

I think the most interesting comment was by one of the authors of the paper which I quoted in my last response. The fact that he is saying that the methodology chosen affects results is very informative. Researchers normally don't criticize defenses of their own papers, and it further corroborates Dr. Bates' and others' claims that the K15 lead author had his thumb on the scale here, pushing for a specific result.

"Suffering lies not with inequality, but with dependence." -Voltaire

"Fallacies do not cease to be fallacies because they become fashions.” -G.K. Chesterton

"In his article, David Rose relies on reports from a researcher at NOAA who was unhappy about the data archiving associated with the Karl et al paper. While I cannot speak to how well the authors followed internal protocols, they did release their temperature anomalies, spatially gridded data land and ocean data, and the land station data associated with their analysis. They put all of this up on NOAA’s FTP site in early June 2015, at the time that the Karl et al paper was published.

As someone who works on and develops surface temperature records, the data they provided would be sufficient for me to examine their analysis in detail and see how it compared to other groups. In fact, I used the data they provided shortly after the paper was published to do just that. While it would have been nice for them to publish their full analysis code online as well as the data, I’m sure they would have provided it to any researchers who asked." (same source)