ccafs falls in the data client camp – its focus is on getting users
data – many rOpenSci packages
fall into this area. These kinds of packages are important so that
scientists don't have to recreate the wheel themselves every time, but
instead use one client that everyone else uses.

CCAFS GCM data files are .zip files with a bunch of files inside. The
individual files are in ARC ASCII format (https://en.wikipedia.org/wiki/Esri_grid#ASCII) –
a plain text data format, but still require painful manipulation/wrangling to
get into an easily consumable format. The files have a .asc file extension.

For each .asc file, the first 6 lines of each file indicate the reference of
the grid (number of columns and rows, corner coordinates, cellsize, and missing
data value), followed by the actual data values, delimited with single
space characters.

There's a related binary format – but its proprietary, so nevermind.

The workflow with ccafs for most users will likely be as follows:

Search for data they want: cc_search()

Fetch/download data: cc_data_fetch()

Reaad data: cc_data_read()

I'll dive into more details below.

Installation

First, install the package.

install.packages("ccafs")

Then load ccafs

library("ccafs")

Search for data

Searching CCAF's data holdings is not as easy as it could be as they don't
provide any programmatic way to do so. However, we provide a way to search
using their web interface from R.

You can search by the numbers representing each possible value for
each parameter. See the ?'ccafs-search' for help on what the numbers
refer to.

When using cc_list_keys(), you'll get not just .zip files that can be
downloaded, but also directories. So beware that if you're going after grabbing
"keys" for files that can be downloaded, you're looking for .zip files.

Fetch and read data

Once you get links from cc_search() or "keys" from cc_list_keys(), you
can pass either to cc_data_fetch() – which normalizes the input – so it
doesn't matter whether you pass in e.g.,

Which gives a raster class object, which you are likely familiar with – which
opens up all the tools that deal with raster class objects, yay!

You can easily plot the data with the plot method from the raster package.

library("raster")
plot(dat)

Caching

For a better user experience, we cache files for you. That means
when we download data, we put the files in a known location. When a
user tries to download the same data again, we look to see if it's already
been downloaded, and use the cached version – if we don't have it
already, we download it.

Of course, CCAFS may change their files, so you may not want the cached
version, but the new version from them. We provide tools to inspect your
cached files, and delete them.

One thing in particular that improved about ccafs was the user interface –
that is, the programmatic interface. One feature about the interface was
adding the cc_search() function. When I started developing ccafs, I didn't
see a way to programmatically search CCAFS data – other than the Amazon S3
data, which isn't really search, but more like listing files in a directory –
so I just left it at that. During the reviews, reviewers wanted a clear workflow
for potential users – the package as submitted for review didn't really have a
clear workflow; it was

Know what you want already (cc_list_keys helped get real paths at least)

Download data

Read data

Which is not ideal. There should be a discovery portion to the workflow. So
I decided to dig into possibly querying the CCAFS web portal itself. That panned
out, and the workflow we have now is much better:

Search for data with all the same variables you would on the CCAFS website

Download data

Read data

This is much better!

As always, reviews improved the documentation a lot by pointing out areas that
could use improvement – which all users will greatly benefit from.

To Do and Feedback

There's probably lots of improvements that can be made – I'm looking forward
to getting feedback from users on any bugs or feature requests. One immediate
thing is to make the cache details more compact.