Wednesday, June 04, 2014

In the spirit of Deep Climate, Anonymous, as he wrote to Eli, has penned an analysis of how Richard Tol cooked and indeed, it is McIntyre class, although from the nature of the mistake, it may very well have been hubris and not malice. Dickie's misbegotten analysis starts from Cook et al's description of their procedure, and just to show that Eli is an honest card shark, why not start with Tol's description first of what he did, and then of what Cook et al did. At the end of this it should become clear that, well, Eli will leave that to the bunnies. From Tol's blog

According to Cook et al.,
each abstract was assessed by at least 2 and at most 3 raters. In fact,
33 abstracts were seen by only one rater, 167 by four raters, and 5 by
five.

Eli will not get into why at this time that of the 12,000 odd abstracts 33 were seen by one rater, but bunnies might ask if those 33 had something about them and whether they were included in the published ratings. Tol rants on

If the initial ratings disagreed, as they did in 33% of cases,
abstracts were revisited by the original raters. In 15.9% of cases, this
led to agreement. In 17.1% of cases, a third rater broke the tie.

A reported error rate of 33%, with 2 ratings and 7 categories, implies
that 18.5% of ratings were incorrect. 0.6% of abstracts received two
identical but wrong ratings. 2.9% of ratings are still wrong after
reconciliation. 3.2% of ratings are wrong after re-rating. In total,
6.7% of reported data are in error.

At this point Dickie goes totally off the rails. That there were disagreements in the ratings is clear, such disagreement is to be expected when one imposes an ordinal value on a continuous one. For example, if bunnies measure a continuous value of 3.5 + 0.1 imposing an ordinal value will in half the cases result in an ordinal value of 3 and in the other half of the cases in an ordinal value of 4. In this case it is not surprising that in one third of the cases there was a disagreement amongst the raters.

Where the separation between ratings is only one unit these are not disagreements in any meaningful sense and anyone who claims that they are is fooling himself or attempting to deceive the reader. Those perhaps interested in cross tabbing can look at those from Brian's survey of 666 (yes, Eli reads the Old Testimony) abstracts selected at random from those published between 2002 and 2007 (more about the prequel here).

Tol shows a figure indicating how in the reconciliation and re-rating processes, ratings changed. For 92% of the cases the ratings changed by one unit.

Anonymouse describes Tol's mathturbation

Tol (2014) argues reasonably that from the number of disagreements, the error rate in the initial abstract ratings was about 18.5%. He further argues that if the same error rate applies during the reconciliation process then 6.7% of ratings will still be in error after reconciliation, implying that 11.8% were corrected during reconciliation. He assumes that the remaining errors are equally distributed among categories, a claim which is problematic but which will be assumed for the remainder of this analysis.

Eli disagrees with this on two grounds. First, as mentioned previously, these differences, and certainly the differences of one unit are in no way errors. Second, the assumption that the remaining differences are equally distributed among categories is more than problematic. It is WRONG and the data showing that it is WRONG have been made public. Anonymouse continues

In Tol’s analysis the 6.7% of error ratings are redistributed to other categories in proportion to the corresponding proportion of shifts in the histogram. Shifts which would move the abstract rating outside the 1...7 range leave the abstract in the most extreme category.

This operation on the error ratings may be represented by the matrix S

Which can be used to produce the Tolian 90% version of the Cook, et al 97% result viaT = (1-0.067)I - .067S using the 6.7% disagreement rate.

multiply the T matrix by the vector of the ratings in each category and you get Tol's results, which, in spite of Richard's whining, is a pretty good check of the procedure. Although the elements (1,1) and (7,7) appear irregular (Mark R in the comments at Tol Cooking, thinks they should be zero, and Eli did at first, they are correct, as can be seen by adding all the columns (and rows) which sum to unity as they should. Richard is spewing snot about this.

As Tol points out in his blogspot, AS does get the wrong matrix S.
However, his calculations have been redone with the correct matrix S,
and yield essentially the same results:

- If one were to expect
Tol's approach to be applicable to the only solid evidence there is (the
records of the reconciliation), the approach requires that the initial
distribution have negative numbers in one category, prior to
reconciliation; and

- the results of Tol's projection bear no
similarity to the statistics they seek to model: How does a 2:98 split
turn into a 55:45 split? via the power of Tol's assumptions! and this is
for the category that comprises 2/3 of the papers that were studied.

The
Tol comment method predicts that it changes from 96.73% to 87.44%
(versus observed 97.06%). Working backwards, Tol's correction requires
that the pre-reconciliation consensus should be >100% (as pointed out
by Anonymous).

I'm sure that Professor Tol will realise and retract his paper or, at the very least, submit a quick correction.

added by Eli for convenience Mark R's S matrix

0.0

0.429017350

0.030959032

0.001167679

0.000468604

0.000000000

0.000000000

0.916281027

0.0

0.415735568

0.031060252

0.001171509

0.000487329

0.000000000

0.068994531

0.523180399

0.0

0.417094816

0.031162137

0.001218324

0.001038422

0.006731174

0.039394667

0.506983240

0.0

0.418462980

0.032407407

0.002596054

0.007993269

0.003843382

0.038175047

0.508640822

0.0

0.435185185

0.069055036

0.000000000

0.004564016

0.003724395

0.038299860

0.510309278

0.0

0.927310488

0.000000000

0.000000000

0.004422719

0.003736572

0.038425492

0.530701754

0.0

Which agrees with Tol's. This is somewhat hidden in the URL that Neal gave. Bunnies have to download the Excel spreadsheet http://www.sussex.ac.uk/Users/rt220/Consensus.xlsx and also http://www.sussex.ac.uk/Users/rt220/tcp_allratings.xlsx which are under the headings Data and Graphs on Abstract Ratings and Individual Ratings
--------------------------------------------

The problem being that if you use the inverse, the percentage agreeing with the IPCC consensus is 116% and the number of papers rejecting the consensus at level 5/7, 7 being outright Dragon Slayer territory, is -555 which is the new number of the beast. The two errors in the S matrix will not change that much

What to say, what to say. . . First of all shifts that would move the abstract out of the ordinal range should have a probability of zero. This unjustifiably inflates the number of abstracts at the extremes. Second, Skeptical Science has just published it's full response, and bunnies can see how Tol's assumptions lead to Tol's conclusions

89 comments:

As Tol points out in his blogspot, AS does get the wrong matrix S. However, his calculations have been redone with the correct matrix S, and yield essentially the same results: - If one were to expect Tol's approach to be applicable to the only solid evidence there is (the records of the reconciliation), the approach requires that the initial distribution have negative numbers in one category, prior to reconciliation; and - the results of Tol's projection bear no similarity to the statistics they seek to model: How does a 2:98 split turn into a 55:45 split? via the power of Tol's assumptions! and this is for the category that comprises 2/3 of the papers that were studied.

The Tol comment method predicts that it changes from 96.73% to 87.44% (versus observed 97.06%). Working backwards, Tol's correction requires that the pre-reconciliation consensus should be >100% (as pointed out by Anonymous).

I'm sure that Professor Tol will realise and retract his paper or, at the very least, submit a quick correction.

MarkR,I'm sure that Professor Tol will realise and retract his paper or, at the very least, submit a quick correction. That's very funny, but you really should put a smiley face or some kind of emoticon at the end of such statements, or else people might think you're being serious.

ATTP: I have much more hope than you! Everyone makes mistakes and it's embarrassing when it happens, but it's much more embarrassing in academia to deny them forever rather than slip out a quick correction.

The climate debate is loud, tribal and aggressive sometimes and I imagine Professor Tol could be feeling defensive at the moment. Snide comments are fun and we all give into the temptation, but they could trigger stronger emotive responses that get in the way of fixing the damn problems with your maths.

Tol has shown that he can overcome tribal feelings by his repeated statements that there really is a consensus, despite his 'sympathisers' (GWPF, US Republicans) not wanting to hear anything about it. So I stick by that statement and don't think it needs an emoticon at the end. :)

MarkR,You are, of course, quite right and my comment was indeed snarky. At the moment, however, I'm finding it hard not to make such comments, but I would happily apologies and feel duly embarrassed if my comments were shown to be unfair. I shall do my best to follow your much better example - although I may not always succeed :-)

Dude, can you point out this debate to this informed and numerate individual, because, well, I just don't see it. What is pathetic is claiming there is still a debate.

Go ahead, make my day, debate all you want, and I'll show you some snark that will shut you down in a second. That you are even debating Tol's masturbation is pretty pathetic as well. Methinks you need a much thicker skin.

Of course there are debates, perhaps even more than ever. Debates are everywhere, in the climate blogosphere and elsewhere. Even if these debates are not justified on scientific ground, they still exist.

But debates over what? That is the question. What debates should we have instead? That is another question.

Do you realize, dear Thomas, that your argument is of the same form as Bjorn's?

Incidentally, this is the same as Judy's:

I find the topic of abrupt climate change to be more more interesting scientifically than AGW, and potentially of greater societal significance (although one key question is the potential role of AGW in triggering an abrupt event). IMO much more scientific effort should be focused on the topic of abrupt climate change.

http://judithcurry.com/2014/06/04/explaining-abrupt-climate-change

Commenters from the private sector should beware equivocation over the word "debate".

Well, for starters, there was "What is pathetic is claiming there is still a debate." Then there was "That you are even debating Tol's masturbation is pretty pathetic as well." This claim was followed by a list starting with "Epidemic Diseases," which was a response to what we should debate instead of Tol.

In other words, the argument goes like this, paraphrasing:

(1) Debating Tol is stupid;(2) There are more important things to debate;(3) Here are some more important things;(4) There is no debate anyway;(5) We ought not debate Tol.

This argument is of the form "we should rather do B instead of A (i.e. what we're doing right now) because B is less important than A.

Compare and contrast:

We'd rather fight poverty instead of debating over mitigation.

We'd rather study emergent properties of natural variability than debating over AGW.

We'd rather debate just about any important societal topic than to debate Tol.

Why do we care? There are a very few smart guys out there that get things correct, and a lot of dumb guys out there get stuff wrong, frequently.

I would rather have the considered thoughts of 3 really smart guys, than the ramblings of 97 not so smart guys. Science does not work by vote. The smartest guy in the room may not have all the facts, but if you have the 3 smartest guys come to a consensus, you likely have the right answer. If you wait for a consensus of 97%, then you can be sure that you have the right answer, but it is too late - your prey has moved on or the monsoon has passed and it is too late to plant rice. The species has traded certainty for time to act.

Mother Nature is running the program and she is only rewards species that are able to tolerate very high levels of uncertainty. Mother Nature punishes species that demand certainty.

Aaron,True, scientific truth is not a matter of polling--but then scientific consensus is not either. If you want a better metric for measuring scientific consensus, look at publications or better yet, citations. At some point and idea, technique or theory becomes so indispensable to an area of study that you cease to publish meaningful work if you don't embrace it. Then you have consensus.

No, it's because I can't index blogspot comments without looking a the HTML source for comment indexing tags if they exist. And guess what, I don't feel like wasting my time on you more than I already have. It is ... unproductive.

If you think you are anybody's peer around here, review it. Can't find it? Look for it. If not, I posted a whole set of problems for your PhD thesis. Take your pick and get back to me. I will be quizzing you.

Please forgive me for not bothering with the rest of your spewage. It's ... too ... lame. I give anything you say as much credence as I do Richard To, Judy or Roger Pielke (Jr or Sr.). That kind of nonsense might work for corrupt and ignorant congressmen and woman, but hey, I've got just enough science and math to see through any bullshit you or any of your other cronies throw my way. So good luck with the time wasting. You are a waste of time.

Here goes. You assume that Eli's is a place to solve first, second, and third world problems. That this thread about Tol is exactly where we should be talking about that. That "debating" Tol's ideas somehow "avoids" to contribute on these problems.

These concerns follows the same kind of logic as Bjorn's argument against mitigation.

Please own it.

***

Here would be a comment to help you solve first, second, third problems: please curb your enthusiasm regarding Eli's. This is not the place where this will happen. Not the way you envision it. You can do better than commenting here.

Here would be another one: to think that everyone should always try to solve first, second, and third problems the way you envision it is just silly. There are lots of people who tackle these problems already. There are many ways to solve these problems, and paying due diligence to what Tol writes may even have an impact, however indirect or remote it might be.

> Tol's main argument has been that below about 2C, the impacts of warming would be positive.

Here's the short summary of the Copenhagen Consensus:

Global warming is a major global challenge, which could increase food and water insecurity for many millions of people in the developing world and cause trillions of dollars worth of damage to the world economy. An affordable, cost-effective package of measures would not only start the process of reducing greenhouse gas emissions, but would also enable other Copenhagen Consensus challenges to be properly addressed.

Many people believe this is the world’s biggest problem and solving it should be the top priority. This research shows that even the most extreme mitigation efforts (reducing carbon emissions through taxes) would not guarantee the worst damage of warming. This is in line with recent IPCC findings – but the paper goes further and fills in the gaps about the mixture of spending that would provide value for money. One solution it proposes is mixing adaptation and mitigation approaches with greater research and development into greener technology resulting in high returns.

http://www.copenhagenconsensus.com/expert/richard-s-j-tol

Richard may very well be a waste of time to bunnies, but dismissing him from one's armchair won't prevent him to waste more time in the world.

Another item I left off the list purposely was plain old weather. This of course led to homes and automobiles and thus the problems, which of course are related to unprotected sex, overpopulation and is intimately related to irrational religious beliefs. lol. Certainly one of the best things to ever happen to the human race is latex. It's just too bad two of the major irrational religions (Catholicism and Islam) aren't wise to this.

From the top of my hat, I'd add to that list malaria (which may have killed half of the human species to date), eternal corporations, inheritance (in law; perhaps in OOP too, but I'm tempted to add OOP in general), and populist rhetoric.

I'm not looking for anything that specific, and it falls under the phrase 'epidemic diseases'. Antibiotic resistance is a category by itself since it concerns bacterial response (although not viral response). Nowadays one can hardly claim ignorance unless living wild in the jungles.

Botkin's testimony contains a very similar argument than Thomas'. It does not mention that Richard is a waste of time.

But sure, one can certainly argue that Richard, who was a lead author to the WGII, who testified to Congress not long ago, who is ranked among the Top 100 economists in the world by IDEAS/RePEc1, who is among the 25 most cited climate researchers according to Google Scholar, and who is the most cited scholar in the Stern Report, is a waste of time.

Anyone (except you?) can see that "Eli invests time so that otters won't have to:"

followed immediately after and was clearly a reference to "Richard may very well be a waste of time to bunnies, but dismissing him from one's armchair won't prevent him to waste more time in the world."

You should really change your pseudonym because you are making a mockery of logic and language.

Your list of 'problems' is so vague as to be useless. It's ... tolish. If that's all you can up with then you are indeed in very serious trouble. Good luck with that.

Here's a couple of solutions to wrap your naive and illucid mind around - resonant ZT enhancement and topological degree of freedom reduction. You are just so special. Tol is just too funny, a caricature of all that is wrong with academia, joining the ranks of Judy, Bjorn and the Rogers, and of course, then there is you, the most eminent pseudoscientist. Show me your license and your CV!

> Anyone (except you?) can see that "Eli invests time so that otters won't have to:"

Anyone can see that "otters" does not imply all otters, Anonymous. It did not imply Thomas, for instance, as I already told you and which you failed to acknowledge. Grammar suits you better than quantifiers.

Would that be useful to you?Perhaps not, as you may not need any list at all:

> The problem I see with transitioning to a completely carbon free or carbon neutral energy economy is that, as wonderful as that may be, it is only one of a myriad of horrible problems that are self evident at the extinction level. The major problem is the use of energy, Thus I have come to the conclusion that complete evacuation of the human species from the planet is the only solution that will yield anything even vaguely resembling an early Holocene era planetary ecological and biological diversity. And indeed, I have worked out the necessary global technological and engineering actions required to produce that desired result. Your opinion on the subject of future evolution of humanity may diverge from my own, of course, but it is a subject I have studied deeply.

Sure if you are an idiotic policy wonk. My issues are solar irradiance, composted maple leaves and where my food comes from. After my air and water, of course.

Please tell bunnies more about your deep studies.

Why bother when it's free for anybody to read. The bunnies know all this stuff. Why are you so concerned about the bunnies?

They eat grass, they don't even eat maple seeds. Or maybe they do but I've never seen them eat any.

You get your food from coal, oil and gas. Very bad for you. Very bad for the land, water and air.

Dude, that's all you need to know. Solar irradiance. Atoms and molecules. Get it? Do I have to spell it out for you in SI units, or what? It's not my problem third world people live in third world slums and third world conditions, I've got my own hood and concrete jungle to deal with. And farmer joe and his boys, lol. You do it your way, and I do it my way. I just happen to prefer it the abundant solar irradiance, water and deciduous leaf soil way. Mostly because it works.

Not that's funny. You live on a terrestrial planet with an active hydrogeological cycle and ice caps.

If you have a problem with water, it's your own damn fault. What you have is a problem with the price of water, and the price of transporting it. There is no crisis. What you are doing is screwing up the weather, as an effect of screwing up the climate, and you are utterly destroying your aquifers. Any crisis is of your own making.

Read the list again. The natural problems are way at the bottom. And I added a postdated addition to the top of the list (antibiotic resistance), and put debt at the end of the global financial instabilities entry.

The following update has been left at the page of Richard Tol's dismissal of A.Scientist's brief argument against the validity of Tol's calculation of reassignments:================================Neal J. King7 June 2014 05:15

[Below, I denote the set of abstract counts per category published in Cook et al. 2013 as '(C13)'. I refer to the set of abstract counts, prior to the reconciliation procedure, as the 'ur_count'; this is:

(128, 1844, 5820, 15,940, 108, 30, 18)

Richard:

- It is true that AS got the wrong matrix S; however his approach of defining the matrix T(eps) = (1 - eps)*I + (eps)*S seems to be an accurate description of your calculation. Some of us have taken your invitation to use the correct matrix S', and have substantially reproduced his conclusions.

- To verify the proper S: According to our calculations, your projected correction of C13 is T(6.7)*(C13)

= (92.76, 961.65, 2970.25, 7540.56, 329.10, 37.77, 11.92).

(Unfortunately, there will be a slight difference from your calculated numbers: In your spreadsheet calculation, we believe that bias!B20-B26 should refer to the normalized column O, not the raw histogram in column N.)

- Based on our version of matrix S, we calculate the inverse of T(11.8) = T_inv(11.8). In accordance with AS's argument, if it makes sense to apply T(6.7) to (C13) to project the corrected counts, it should also make sense to apply T(11.8) to the ur_count to obtain (C13):

T(11.8)*(ur_count) = (C13).

But that implies:

(ur_count) = T_inv(11.8)*(C13)

= (10.7, 854.3, 2745.9, 8879.1, -559.5, 7.0, 6.5).

- Comparison with the (ur_count) stated above shows that this is wrong; but with a negative number of papers in category 5, it is clearly absurd. This means that T(11.8) was not the process applied to the (ur_count); but if T(11.8) is not applicable to the ur_count to obtain (C13), why should T(6.7), which is what you have used, be applicable to (C13) to produce the projected correction? For T(11.8) is constructed on the same lines as T(6.7): they only differ in the values of correction strength, which you have estimated as 11.8% and 6.7%.

- Or in other words: If for some reason you don't believe the S matrix approach applies to the ur_count-to-(C13) transition, please explain why you apply it to the (C13)-to-correction transition. Particularly since the ur_count-to-(C13) transition is the only one that is possessed of actual data, as opposed to speculation.

- Or yet in other words: How do you justify a model that concludes that a 55/45 split is an appropriate representation of an actual 2/98 split - and for a category that comprises 2/3 of all articles?

The linear algebra, matrix manipulation and statistical transformations aside, the take away I'm getting from this lesson, besides the fundamental issues of peer review, is that one must or should choose your problems wisely (or in the cases under discussion, urgently) and just do the best job you can on them, and take your knocks when they come. Peer review via crowd sourcing and enthusiast forums works, I've seen this just recently with that satellite Cowing and Wingo have been working on. This is most definitely a new paradigm.

Tol's problem isn't urgent, but correcting it was, since it was rather late in the process and had been corrupted by the media.

Neal's comment is very interesting, but he still assumes an error rate from ur_count to C13 of 11.8%. Tol in fact assumes a constant 18.5% error count per rating process. With an initial disagreement rate of 33% between first and second ratings, the error rate from ur_count to C13 is 18.5% of 33%, or 6.1% with the remaining 0.57% of errors being undetected errors from initial rating when both initial raters provide the same erroneous value.

I am asking for a recalculation with the correct values. Before doing so, however, I will note that Tol's calculation of the base error rate is incorrect. He assumes equal probabilities for all rating values in the calculation, so that the 2 times the individual error rate minus (1/6 times the individual error rate squared) equals the disagreement rate after initial ratings. However, not all ratings have equal probability. My best estimate is that there is a 52.6% chance that two erroneous ratings will coincide, from which it follows the individual error rate falls to around 17.3%. That increases the rate of initial errors escaping attention to 1.57% of all abstracts, the error rate during reconciliation to 5.71% of all abstracts, and the total error rate to 7.28%. This assumes the average error distribution. Using the correct error distribution by rating may adjust these figures further. In any event, would anybody care to repeat anonymous scientist's analysis with these corrected values.

Tom, do not fall into Tol's (and Lucia, etc.) trap. The disagreements on first rating ARE NOT ERRORS. As SkS points out the disagreements are almost all the result of imposing an ordinal value on the continuum of positions.

Consider what the disagreement rate would be if the value of a set of papers was 3.5 +/- 0.1? 50%.

Eli, we can call the "errors" disagreements and the "undetected errors" accidental agreements if you prefer. However, it makes no difference to the analysis or the final consensus rate found at the end of that analysis. Therefore I do not think the terminological issue is substantive.

More importantly, SkS points out the problem is treating ordinals as a continuum rather than the reverse (See error 14 of the 24 errors). The endorsement classification is achieved by responding to a set of binary oppositions. Does the abstract endorse the consensus, or not? If so, does it endorse it explicitly or implicitly? If explicitly, does it quantify the contribution of anthropogenic factors to recent warming? If it does not endorse, does it reject, or is it neutral? And so on. Partial values have no meaning in this context. An endorsement rating of 3.5, is nonsense. It literally has no meaning in the classification scheme.

What is true is that the determination of the answer to the binary oppositions is often difficult. In such circumstances, many people will get the answer wrong, and indeed whether the same person gets the answer right or wrong may vary based on arbitrary factors (time of day, tiredness, endorsement rating of other recent ratings etc). Further, what determines that the answer is right or wrong may be somewhat arbitrary. It may be resolved, for example, how it would be assessed by the majority of climate scientists, or by the authors of the paper, etc. That means we may not have a final determination as to which answer it genuinely right or wrong, but by the logic of the classification system, it is presumed that a unique endorsement rating is correct for each abstract, and that all other endorsement ratings are incorrect for that abstract. (This is not the same issue as to whether the error of rating a level 1 paper as a level 2 paper is as consequential as rating a level 3 paper as a level 4, or worse a level 5 paper. They are not, as SkS again, correctly, points out.)

Finally, regardless of the truth of the matter, the "errors" discussed in Tol's faulty analysis are internal errors. They are estimate disagreements with the limit of classification from the actual raters as the number of agreements on classification becomes to finalize a rating arbitrarily large. For most initial ratings, the raters got that right, but there were some errors (or disagreements if you prefer), and the approximate rate of those disagreements, and their effect can be estimated. Just not using Tol's S matrix, which rates more as a blunder than as an error.

> The math was all done by otters and Eli merely posted their results.

Indeed, and the other Anonymous should simply have had to post his results on ArXiV, instead of bothering Eli with it. Blogging is so useless. We should leave everything to interpreters of interpretations.

I'm not sure what Musk was doing in the shower, but at least some of it was an excursion into fantasy.

I've said that I've said it before and that I'll say it again, so I'll say it again... Interplanetary colonisalism in any population-effective form is a pipe-dream of the highest order, and we'll have Quiddich on Firebolts as an Olympic sport before we see interplanetary settlement beyond a few shivering, starving and suffocating suicides on Mars.

Thermodynamics says "no", and no business plan has addressed the inescapable energy budget, let alone the ecosystem transplant/compatibility issues involved or the ethics of colonising habitable planets if they happen to already support life. And as evidence of the infeasibility of interplanetary (and especially interstellar) colonialism, I tender the fact that our planet is still absent any little green aliens.

We have one blue marble in the whole universe. It's our alpha and omega, and anyone who imagines otherwise is simply getting in the way of making sure that omega is as far away as possible.

"I'll believe in people settling Mars at about the same time I see people settling the Gobi Desert. The Gobi Desert is about a thousand times as hospitable as Mars and five hundred times cheaper and easier to reach. Nobody ever writes "Gobi Desert Opera" because, well, it's just kind of plonkingly obvious that there's no good reason to go there and live. It's ugly, it's inhospitable and there's no way to make it pay. Mars is just the same, really. We just romanticize it because it's so hard to reach."

Interplanetary colonisalism in any population-effective form is a pipe-dream of the highest order, and we'll have Quiddich on Firebolts as an Olympic sport before we see interplanetary settlement beyond a few shivering, starving and suffocating suicides on Mars.

I'm pretty confident I can file that with numerous other statements such as 'spaceflight is utter drivel', 'if man were meant to fly' and 'who needs more than 640K'. The only thing that will stop this is the collapse of civilization. That is the alternative that you propose.

Good luck with that, fool. Please, by all means, feel free to make a fool of yourself here. Everybody knows superconductivity above 22.5 K is impossible. Now tell me, what is the average yearly number of airline passengers. Quickly now, billions of dollars of revenue are at stake.

I didn't say anything about going to Mars, that was presumptions from you idiots. Let me explain it to you, Charlie Stross is a science fiction author. Is that where you get your freakin science? That explains a lot about the state of the biosphere. Elon Musk built his own rocket engine and rocket. Get over it. You can't stop it now.

Your logic is laughably tragic. Normally I would not even bother to debate someone so naive, someone so behind the curve both technologically and scientifically. It's just a waste of time for me. All I can say is get out there and educate yourself. Start here. I regularly blog titles that I feel are relevant to your problems. I regularly publish essays on these subjects as well. Even better, get yourself an education, and do try not to quantify that with degrees.

I find it very easy to dismiss something long after it has been demonstrated to be both irrelevant and wrong. Richard Tol, in a nutshell. Maybe his next paper will be an improvement, lol (as in relevant, not fraught with errors and not fundamentally wrong). That part I am not dismissive of in the least.

I'm guessing at least Musk and Stross both understand that the moon is a long way and uphill from low earth orbit.

Musk's genius is manifest not so much in replicating what others have done for a half-century, nor in profiting from the publicly funded R&D involved in that, but in gaining the admiration for the same from the people who wrote the checks.

Right, but you could be living on a super Earth with no moon, so count your blessings you happen to be on a relatively small terrestrial planet barely able to support plate tectonics. And uphill is quantified we have numeric and computational symplectic integrators now that can easily handle anything you throw at it. And now get this, the technology we need to do this is pretty much the same as what we need to live sustainably on the Earth. Cryogenic production, storage and distribution, fuel cells, cooling, heat exchangers, solar electric, batteries, propulsion, life support, you name it, we need it cheaper and better. I guess you just missed that I suppose. Besides, you don't seem to understand. You have no choice anymore. It's a done deal. So get on with it. Hint : SLS and Orion in their current form are not 'it'. It seems to me you have a real NASA problem on your hands as well.

Thomas does not always dismiss, but when he does, it's because it's irrelevant and wrong.

You missed 'after it was demonstrated' to be irrelevant and wrong. This is where the smell test comes in. That saves a lot of time. Generally I give crackpots more of my time than people like you. The end result is generally the same,, but you never know, they might be giants.

I don't think you have a very firm grasp of what constitutes credible solutions to biological problems on a terrestrial planet.

I said "it is irrelevant and wrong," which it what a demonstration usually implies. I could have said "after Thomas declares it irrelevant and wrong," to underline the question begged. In either case, Thomas dismissed the topic of this post not because it was wrong, but because he considers it irrelevant.

***

Another list, this time they're called challenges:

1. How can sustainable development be achieved for all while addressing global climate change?

2. How can everyone have sufficient clean water without conflict?

3. How can population growth and resources be brought into balance?

4. How can genuine democracy emerge from authoritarian regimes?

5. How can policymaking be made more sensitive to global long-term perspectives?

6. How can the global convergence of information and communications technologies work for everyone?

7. How can ethical market economies be encouraged to help reduce the gap between rich and poor?

8. How can the threat of new and reemerging diseases and immune micro-organisms be reduced?

9. How can the capacity to decide be improved as the nature of work and institutions change?

10. How can shared values and new security strategies reduce ethnic conflicts, terrorism, and the use of weapons of mass destruction?

11. How can the changing status of women help improve the human condition?

12. How can transnational organized crime networks be stopped from becoming more powerful and sophisticated global enterprises?

13. How can growing energy demands be met safely and efficiently?

14. How can scientific and technological breakthroughs be accelerated to improve the human condition?

15. How can ethical considerations become more routinely incorporated into global decisions?

I think you should worry more about backing up away from the agricultural cliff you've created for yourselves, and into something a little more resistant to all those known agricultural disruption mechanisms and probabilities. Nothing makes humans more ornery and desperate than outright hunger.

The way I calculate it, the probability of a disagreement between two evaluations when the likelihood of "error" is x is: D = 2x(1-x) + 6*(x/6)^2 = 2x(1-x) + (x^2)/6

This breaks down into:a) the likelihood that one rater gets right while the other gets it wrong; plusb) the likelihood that both get it wrong - but agree on the answer (1 out of 6). Probabilities for erring in the direction of each category are assumed equal.c) Tol set this expression, which I will denote as D, to be:D = 0.33. Solving this with the quadratic formula, one infers that:x = 18.5%d) I believe you have an erroneous minus sign in the equation for D, leading you to the anti-intuitive conclusion that a greater likelihood of two raters agreeing in error reduces the remaining-error rate.e) I need to think a bit more about how unequal probabilities affect things. My sense is that it gets ugly.

Eli (8 June, 1:42 am), just over 9% of disagreements between initial ratings and final ratings were larger than 1 point. These, at least, must be considered genuine errors by either the initial rater or the adjudicator. Further, given that large percentage, a significant percentage of those ratings disagreeing by just one point must be considered to be disagreements over papers that were near central values. That means the number of disagreements to which your sorites style argument applies is unknown. In consequence it is not possible to divide discrepancies into those which are genuine errors (even by your standard) and those which are mere discrepancies due to indeterminancy of classification. Further, that sub-analysis is irrelevant except at most for rhetorical points. Consequently it is better to have a single name for both.

Further, as regards the rhetoric, among people interested in the arcane enough to even follow this debate, those with an open mind will happily accept that humans err, and that therefore there are a small number of errors in Cook et al. They will even accept, if they follow the arguments, that classification is difficult so that there will be variation in classification at the margins, and that considering that Cook et al achieved a surprisingly low error rate. Before they get to that point, however, they may well be put of by people insisting that what they can clearly see are errors are not.

Neal, if x is an error rate of 18.5%, than 2x = 1.63, and hence is not a probability. Adding a positive number only makes the problem worse, so your formula must be wrong.

It is best to start by considering the ratings to be binary choices so that if it is not correct, then it is an error. Assume, further that all errors are independent so that no two raters make an error on the same abstract. Then, as any error creates a discrepancy, there will be 2x discrepancies where x is the error rate. Errors are not independent, however, so we must subtract the number of occasions when both raters make an error, ie, x^2, so the base formula is:D = 2x - x^2

This formula can be obtained algebraicly from the rate of agreements (=1-D = (1-x)^2).

The situation is not binary, however, so two ratings both being incorrect do not necessarily agree with each other. With six equally possible errors, that gives you an error rate of 2x - (x^2)/6.

I had another look, and the discrepancy is because Tol uses yet another formula. Using my formula, and equal probabilities the error rate estimate is approx 16.7%

Tom C writes: "...those with an open mind will happily accept that humans err, and that therefore there are a small number of errors in Cook et al. They will even accept, if they follow the arguments, that classification is difficult so that there will be variation in classification at the margins, and that considering that Cook et al achieved a surprisingly low error rate....

Tom, well said.

I think the rhetorical vocabulary is what causes many of the points of contention. Since we are dealing not just with an ordinal scale, but a subjective scale, the disagreement/variation between raters/adjudicators can be either viewed as errors or simply as data.

The implicit assumption is that discrepancies > 1 are 'genuine errors'. Why?

If we knew the true value for each abstract then we could say that any deviation is rating error. Not knowing the true value, each rating can be viewed as an estimation of the true value. And rather than a constant measurement (rating) uncertainty for all abstracts it could just as easily (and more likely to my mind) be that some abstracts have a greater measurement uncertainty than others.

By this latter understanding no rating can be erroneous unless we know the true value and the uncertainty of measurement (rating).

The notion that a disagreement between ratings indicates an error on the part of the raters is full of assumptions that I don't think can be justified.

Of course you prefaced the quote I used to start this comment with, "...among people interested in the arcane enough to even follow this debate..." and now I've let the cat out of the bag -- that I'm still following it :)

Tom:I misread my spreadsheet. Let's try again:a) The probability of error by one reader is x. If two independent readers, with the same error probability of x, disagree, that has probability of 2x*(1-x), because:- one reader must make an error: probability x;- the other reader must not make an error: probability (1-x); - the readers can switch right/wrong roles.Put it all together, and the probability of one reader (out of 2) getting it right and the other getting it wrong is 2 * x * (1-x) = 2x(1-x) which is never greater than 1/2.

b) Another way in which two readers can disagree: they both get it wrong, but they judge different categories. Assuming for simplicity that the error would be equally in any of 6 directions, the probability would be:x for one error;x for the 2nd error;5/6 that they don't agree on the category;giving 5x^2/6 probability that they both get it wrong and disagree anyway.

Thus, the probability that the two readers disagree is:2x(1-x) + 5x^2/6 = D.

For D = 0.33, this gives x = .185

Correspondingly:c) the probability they both get it right is (1-x)*(1-x);d) the probability they both get it wrong, but agree on the evaluation is:(1/6) * x * x = x^2/6.

Rabett Run

Subscribe Rabett Run

The Bunny Trail By Email

Contributors

Eli Rabett

Eli Rabett is a not quite failed professorial techno-bunny, a chair election from retirement, at a wanna be research university that has a lot to be proud of but has swallowed the Kool-Aid. The students are naive but great and the administrators vary day-to-day between homicidal and delusional. His colleagues are smart, but they have a curious inability to see the holes that they dig for themselves. Prof. Rabett is thankful that they occasionally heed his pointing out the implications of the various enthusiasms that rattle around the department and school. Ms. Rabett is thankful that Prof. Rabett occasionally heeds her pointing out that he is nuts.