Chugging Along

Steve Mosher has continued his work on GHCN station global temperature reconstruction package in R. The most recent version is documented in his blog here. He has developed an elegant set of commands for computing global temperatures in R and is working on release of an R library package for that purpose. Recently he incorporated a method from Berkley which Tamino posted. Steve has decided to work entirely in the global temperature datasets on his blog and originally I wasn’t sure if he would keep going or what it would turn into. I check out what has been happening from time to time and got tired of having to google him to find it so I placed the links on the right. He appears to be doggedly developing a very complete package of commands which will process the raw data by a variety of reasonable (and documented) methods.

10 Responses to “Chugging Along”

Average birdfeathers. Add x megajoules to a cubic mile of air over the Sahara. Note temperature change. Add x megajoules to a cubic mile of air over the Amazon. Note temperature change. Add x megajoules to a cubic mile of air over Antarctica. Note temperature change. Add x megajoules to a cubic mile of air over the South Pacific. Note temperature change. Average the changes to within 0.001K. This is ‘y’.

Now, tell someone competent in atmospheric physics that you added 4x megajoules to 4 separate cubic miles of air and raised the average temperature by y degrees. (Give him only the y figure.) Ask him to derive x, to within 0.001mj or so.

Steven Moshersaid

Looks like Roman is plugging away at getting some code ready, if it’s as clean as tamino’s it will go in in a day.

That will leave

1. The actual Berkeley method with weighting. they are currently working in Matlab (yuuuk) BUT I saw their R programmer
at a local R event, so maybe she is boning up to do the matlab to R conversion. die matlab, die
2. Giss… YUUK doing Giss in R will be ugly city. I dont see why they just dont adopt CCC. They should at least use
tamino station combine instead of the reference station method. RSM needs to die.
3. Nick stokes: Nicks is a bit more of a challenge since he’s wrapped a bunch of stuff and options and job control things
in. He’s built more of an application in R ( I tried this early on– ) His code would require a lot of work to turn into
an Api.

If people are wondering what I’m doing its actually building a bunch of functions that are designed to work on spatio/temporal
data.

In class terms.. consider the class of objects we call time series:

1. Data : vector of data
2: time: : vector of time
3. frequency etc

To that class we add the following elements to make a super class

1: data units (C) this is for temperature
2: time vector is regular ( all time from start to finish are present
3: series name: every time series has a unique Id ( the station Id)
4. Time series position: these are located in space
5. metadata: still TBD how to handle this since #3 is a DB key

Then on that class we define methods. Most of these methods will just be clones of basic R methods on the underlying
classes, so window for example will just work, aggregate just works, basic math operators just work. Then we add
methods, so Roman’s approach is just a method on that object.

5 is the biggest hurdle design wise.. right now there are two different objects.. and they need to communicate or pass messages
about changes in each other. kinda sorta

Mark Tsaid

Steven Moshersaid

cause it’s not open. I imagine what I will have to do is get octave ( is that it?0 and then run their matlab in that enviroment, trust that it gives the same answers, then use that version to test and verify the r implementation.

as far as a language goes itlooks pretty reasonable, but I’m not in a position to fork over money so that I can do volunteer work

#5 Matlab is nicer than R by far IMO, but it is $$ and therefore not as useful as open source for the kind of openness that science deserves. I’m thrilled to see high quality work being published on line for this purpose. It would have been a very important tool 3 years ago, and my guess is that with some verification these Steve M R functions may find use in publications down the road.

Garysaid

Mosh, that item #5 seems to me to be crucial. Need to know those station moves and changes. Yeah, in the long run it averages out, but for nearby station comparisons for UHI effects and such it’s important. Good work you’re doing. Any possibility of somebody writing a web-accessible front-end on the R engine so we who don’t have the time to get up to speed with R can work with the data?

Steven Moshersaid

I will look into a web front end. there is a server version of R. Ideally, if this is done properly one would just drag and drop a series of boxes into something that looked like a filter graph ( err get the idea) and then the data/graph pops out. integrating R and web is a ongoing thing, usually guys have java front ends..

the other thing I would look into doing was running stuff from a web page and then populating a GOOGLE fusion table.

I’ve spent a ton of time integrating Google earth and other things… Now perfecting all that into a USEABLE fool proff system is another thing. That takes patience and testing and shit I think I wrote the main code for this at least 10 times.. 10 different ways at it.

WRT#5. ya, I have a bunch of work to do there. how I handle metadata is going to be critical. I have NOT found a system/package that I am happy with. So for now the metadata sits in a R data.frame.. kinda ugly

Steven Moshersaid

I took a look at the code jeff Posted and while it was clear, it was not clear enough for me to clearly explain or integrate.
So, I did a couple things: I changed some functions to output /input the kind of object Roman likes ( a mts) and I asked him to have a go at making the code a bit more accessible. I do have to understand the stuff i integrate to a certain extent. While he does that I will also start my own rewrite of the thermal hammer.. again just to get the feel
of whats going on.

Nicks code: I havent asked Nick yet. First I need to finish going thru his code. he has restructered it from when I first ran it and i’ve got no clear sense of it anymore ( my fault not his) He does some very clever things, but also
does some annoying things ( uses ; at the end of lines)