Plotting a Diurnal Profile with Python and Pandas

A diurnal cycle is anything that occurs daily. In atmospheric chemistry, a diurnal profile displays the trends in a given gas concentration over a period of time at each time throughout the day (makes perfect sense when you see the plot). They are great for visualizing how a certain gas changes in the atmosphere throughout the day (think ozone near a huge interstate interchange).

Plotting them is pretty easy with python and pandas! Here is how you do it:

Read in your data

The pandas library comes with several functions to easily import data from csv, excel, or txt files. There are also several others, including reading from json which will come in handy when I finally get around to working with the EPA API's. I will show how to read in some data from an excel file, since that is currently what I have to work with.

Munge your data

To plot a diurnal profile, we are going to need to group our data in order to perform some statistical analysis of the data. It is important to define a resolution you are satisfied with when you perform the next step; I am fine with minute resolution as seen below.

Create a new column that contains timestamps in the form H:m

data['Time'] = data.index.map(lambda x: x.strftime("%H:%M"))

Group the data by its new Time column and perform some statistics

data = data.groupby('Time').describe().unstack()

At this point, we have a grouped dataframe with columns containing all the statistics you could ever want (and all we need for our analysis).

Set our new index to be of the datetime format for easy plotting

data.index = pd.to_datetime(data.index.astype(str))

Plot your data

Now that our data is properly munged, we can go ahead and plot (fun!). The first plot we will create is a simple diurnal trend showing the mean concentration of the gas (or particle!) throughout the day. The second plot will build on the first one to include the inner and outer quartiles with shading in between (pretty!).

After adding these lines, you should end up with an even more beautiful plot as seen below:

Notes

Python is awesome. Pandas is super-awesome. Science is the greatest! I will upload this code as a github gist as soon as I get around to it. Also, I plan on adding this to the (someday) awesome atmospy python library for the atmospheric sciences.