Code Project: Use weather for wallpapers

Posted at 5:09pm on Friday June 26th 2009

Not all information on the web is static, connected in a meaningful way, or even as cool as it should be. Which is why one of the post-web 2.0 movements of note is the mashup - the repurposing of data from the web into new and exciting forms. Welcome to the world of data punk.

As we start on our journey of discovery, let's do something easy, such as changing our desktop background depending on what the weather's doing. There are already a number of desktop apps and widgets that provide this service, so this data should be freely available on the internet somewhere.

Sure enough, a quick search for "weather API data" provides a whole host of links. Our criteria are: the API must be easy to understand, cover as much of the world as possible and have some reasonable documentation. This rules out a couple of promising sources, mostly on the documentation front - yes, it's possible to spend your time and effort making sense of the data or how the API works, but why would you if someone else provides both the data and a guide on how to use it?

After much fiddling about, it seems that Yahoo's weather service fits our needs well. It's pretty straightforward, and has enough documentation to get us started without much effort. A bit of digging around yields http://developer.yahoo.com/weather, which provides plenty of detail and some examples of how to use the service. Bonus points to Yahoo!

Determining location information is as simple as using Yahoo's online service and reading the URL it provides.

The Yahoo method of working is to append a location identifier to the end of a URL. The service will then provide an RSS feed of the weather data for that area. This is useful in some respects, because it means we can try it out without actually writing any code. It should also be easy enough to find the location code - the documentation suggests just going to the main weather page, typing in our city and taking a good look at the URL it takes us to.

"Bath, GB" takes us to http://weather.yahoo.com/forecast/UKXX0637.html, so our location code is UKXX0637. For some reason, this comes up as Avon Park in Yahoo (perhaps that's the weather station's name?). That oddness aside, we now have our location information. The instructions also say that we can append a value to get temperatures in Fahrenheit or Celsius, so we've chosen to add the Celsius option u=c to our URL. We can now test this URL to see that it works, still without writing any code.

We can check that our submitted URL produces an RSS feed in Firefox, which does a good job of rendering it too.

Feedparser

Okay, so now that we know the information we want is available as a convenient RSS feed, how can we get that into a Python script and decode it? If you open the above URL in Firefox and choose View > Page Source from the menu, you will see that there's quite a lot of data there, plus all the headers and so on.

We could construct a parser that would take this raw information and spit out the bits that we want - but, as is usually the case, someone has already done it. There's a library module for Python called Feedparser that will build a Python object out of an RSS stream for us to fiddle with. You're best off installing this module through your usual package manager, since things can get a bit messy otherwise.

Why Python?

You can, of course, write scripts and applications to crunch web data in pretty much any language, so why are we using Python over C#, for instance? There are several very good reasons. Python is a simple and straightforward language that is easy to write code in and, more importantly perhaps, easy to understand when you read code.

It has excellent capabilities for using and manipulating text (which a lot of our data is going to be), it is cross-platform and there are a huge number of helper libraries available for all sorts of web services and protocols. Using Python and a few libraries, you should be able to bash out a working application or script in no time.

With Feedparser installed, it may finally be time for some coding. Run Python from a shell first, so we can see what we're dealing with. You will be taken to the interactive Python shell where we can start by entering:

The result of this last line is a long spewing chunk of characters, which represents the feed. Fortunately, although it looks like a collection of random terms strung together with a lot of brackets, it is in fact a reasonably well-structured object. To prove it, try entering this in your Python shell:

This simple loop runs through the objects contained in the data object. One of the great things about Python is that it is very easy to manipulate objects, and even get them to tell you a little bit about themselves. For example, here we might want to know exactly what type of object we're dealing with:

>>>type(data)
<class 'feedparser.FeedParserDict'>

Well, that helps us out a little. You'll probably have come across a dictionary object if you've used Python before - it simply stores data as key = value pairs. The objects we got listed out before are the keys in this case. If we browse through the Feedparser documentation, we'll get a bit more background on what exactly these keys hold, since they're common for the Feedparser object. The important one for our purposes is the entries key. This contains a Python list object of the individual feed entries, which form the actual content of the RSS feed.

Python lists start with an index of 0, so to reference the object of the first entry, we would use:

Once again we have a dictionary object with key and value pairs. This time they're defined by the XML structure of the feed itself, so there is no module blueprint for this object, although the items are documented on the Yahoo site. After a bit of investigation, it seems that the summary is going to be the most useful to us, because it contains the current conditions including the temperature.

The slight problem is that the data we need isn't nicely contained in just a single field. This particular slice of the feed is formatted as HTML for rendering on a web page, but we just want the text.

Regular expressions

There's no place to hide from regular expressions (also known as regexes) - these will become increasingly necessary in your quest to mash up the world. A regular expression is just a sort of supercharged way of doing search and replace, and although it looks like an arcane form of glyph-based art, it isn't that difficult to understand. In this case, all we want to do is remove all those unpleasant HTML tags. Although we could probably extract the data we want without going to this trouble, it will become useful later on if we want to adapt this script to handle different sources.

We can import the Python re module (it's included as one of the standard libraries, so there's no need to download anything this time) and put it to work on that text. We don't have the space in this tutorial to explain in detail how regular expressions work, but check out the Regular Expressions box below for more information.

Regular expressions

Regular expressions crop up all over the place. They can be pretty simple, or fiendishly complex. In short, a regular expression is a group of symbols that represents a particular grouping of characters in a string. There are also special characters, such as full stops that can match with any character. In addition, there are lists, groups and even operators, so matches can be grouped together - you can probably come up with a regular expression that covers all the bases for searching.

When you're starting out with building patterns, it's best to use some sort of tool to help you check that you're matching patterns correctly - a single misplaced character is all it takes to cause a catastrophe! One of the better tools is the online regex builder at http://www.gskinner.com/RegExr/desktop. Paste in some sample text and test out your pattern-matching skills. You may also like to visit the regex documentation at http://docs.python.org/library/re.html.

For our expression, we want to match anything enclosed in the 'greater than' and 'less than' signs used as markers for HTML tags. This is pretty easy; the expression would be a <, followed by a pattern for any character, repeated any number of times, then a .+? and > to end. The +? match is the same as +*, but it's a lazy match, meaning it will match the shortest valid string, which is what we want - everything is between <>, so we'd be left with nothing otherwise.

So, now we have the text without the HTML tags, but it still has line breaks. We could use the standard string module to split this into a list, but as we already have the re module loaded, we might as well use that. To search for the new line character, we'll have to tell Python that we want to use a raw string value by placing an r at the front of the string.

As you can see, we have made the assumption that the third element of the resulting list will contain the string we want. To extract the temperature from this, a further regex could be used to match just numbers in that string.

A final step would be to use Python's built-in type conversion to change the string containing the temperature value into an integer for easy comparison. In a real-world script we might combine some of these stages for efficiency, but this is a fairly low overhead application and it does gain some clarity from being broken down into steps. Now all that remains is to fulfil our initial promise of changing the desktop background depending on the weather.

A change in the weather

How will we change the background? Well, we'll just have Python call an external command to do it. For a Gnome backdrop, you can change the image by calling the command gconftool-2, which sets the environment variable that holds the location of the desktop wallpaper filename. That's all we need to do. However, because we will have several variations, it makes sense to turn this into a function.

A simple function isn't going to tax your brain too much. In Python there's a simple statement to define the function and its parameters, followed by indented text. And yes, you can even construct these within the interactive shell:

The first line of the function proper constructs a shell command, while invoking the os.system call executes it. When we call the function, the environment variable is changed and the new image is loaded. The filename we have used here is just a dummy - in reality, it would be a good idea to store your images somewhere accessible in your home folder, such as a directory called weather, and name them something easier to match up with the conditions. With Gnome, you can now use SVG images as wallpaper, which means you can create some nice crisp, scalable graphics for your desktop.

Now, we're not sure about you, but we're thinking there should be five images - freezing, cold, mild, warmish and hot. Please adjust for your geographical location accordingly, but we're going to assign these to the following temperature ranges: below 0C is freezing, 0-8 is cold, 9-15 is mild, 16-25 is warmish and above that is hot.

In some languages, you might have a case/switch construction to deal with this. Python doesn't have one, so we'll just have to use a bank of if/elif/else statements as follows:

Of course, this is just a little script and not a full-blown application by any stretch of the imagination, but it could be the basis of one. What we've achieved here is taking data from one place on the web and placing it automatically into our desktop context. We've seen how a simple RSS feed works and how to manipulate objects in Python. There were some scary regular expressions, and we saw Python hooking into the OS calls to execute external commands. These are all things we can build on as we explore the world of web services and bend them to our will.

You could easily extend this little script, maybe by allowing user selection of the location, or by turning the whole thing into an applet. At the moment, it will fail if it can't reach the internet, which isn't ideal, but we'll learn more tricks for that in future tutorials. Be there!

Who wouldn't like a lovely SVG-based wallpaper of pretty snowflakes? This can be yours!

Web data formats

We've established that a number of web applications exist that can serve up helpful data to us. But you don't really expect it to be that easy, do you? There are several different protocols or ways in which they like to provide this data, and in some cases (such as Flickr) they actually provide more than one. To further confuse things, they are sometimes used inconsistently across sites. In the coming months we'll work our way through the top protocols, so stay tuned!