In course of my Ph.D. dissertation research, I used a physics model called TIE-GCM to study terrestrial atmosphere. One way to run this model is via an online interface hosted on the CCMC website, which provides a way to run the model with customized as well as fixed settings. The customized runs take a bit of time since CCMC has to rerun the model with the new parameters. I was in a time crunch to get a particular set of model runs for my dissertation. In particular, the customization I wanted was to get the model results along the observation path of an instrument called SABER (on board the TIMED satellite). This involved way too many mouse clicks and manual selections on their webpage repeatedly (in my case, it was a couple of hundred times!). Hence I wrote this tool to automate the process.

This script was written in python, utilizing Selenium and the Chrome Webdriver. The web interface to download the data (for the model run I had requested earlier) can be found here.

The script is designed to do several things:

Set the latitude, longitude and the altitude ( the trajectory of the satellite)

Set time / date ranges

Select the output format

Submit request

Use a GET request to get the ASCII data file generated by the tool

Notes:

Some try..except.. code has been used to handle time outs (which may occur frequently – depending on the load on the server).

Several variables have been hard coded into the script, and these will have to be changed to match your requirements.

As the script iterated through the latitudes / longitudes, I noticed that I kept getting several timeout error messages. So to fix this, I added the try .... except.. so as to retry the iteration that just failed. If a time out exception does occur, the webdriver instance is restarted and attempts are made until the the next page loads (which is detected by the presence of an EPS Image).

The new pages that loads after click “Submit”, contains an ASCII link containing the data we need.

We assign the link to a variable called “link”.

Following this, we create a file with the same name.

The cookies are then transferred to a variable so that we can utilize the requests library to download the data.

The script checks for a successful connection to the server, and then downloads the data to a file.

# Reopening main URL
try:
browser.get(URL)
# Handling time out by restarting the browser
except TimeoutException:
browser.quit()
browser = webdriver.Chrome()
browser.get(URL)
startTime += 72 # This is based on the resolution you want to use. 72 here refers to a resolution of 24 hours

This bit redirect the webdriver back to the original interface URL, and if it has trouble accessing the URL, reloads the chromedriver. The next iteration will have the startTime incremented by 72 (equivalent to 24 hours)