To go from here to a RSS aggregator is easy. You just have to extend the code above with the functionality to retreive info from several RSS feeds. You may also want to present the info in some other format, e.g HTML via a Yaws page. This however, is left as an exercise for the reader to do.

To go from here to a RSS aggregator is easy. You just have to extend the code above with the functionality to retreive info from several RSS feeds. You may also want to present the info in some other format, e.g HTML via a Yaws page. This however, is left as an exercise for the reader to do.

Revision as of 08:46, 30 June 2006

Contents

How to write an RSS aggregator

Introduction

In the article: is RSS, they describe the various RSS formats and create a simple RSS aggregator written in Python. Inspired by this I decided to do the same in Erlang.

In this example I have been using OTP-R10B-3 release and the
jerl Jungerl start script. By using the Jungerl start script
I automatically get www_tools in my path. This will probably
also make it possible to make the example work in older Erlang releases
since Jungerl also contains xmerl before it was added into OTP.

Getting information from an RSS feed.

Let us use the RSS feed at Slashdot in this example.
The RSS info can be reteived from the URL: http://slashdot.org/index.rss.
We make use of a function in the www_tools package to retrieve the
file.

Printing out the RSS information.

Now we have to extract the information we need from the parse tree. We will write some simple code to do this. But first, let us see at how the result looks like.

Code listing 1.3: Printing out the RSS info

3> <span class="input">myxml:printItems(myxml:getElementsByTagName(Doc, item)).</span>
title: United Kingdom Leads the World in TV Downloads
link: http://slashdot.org/article.pl?sid=05/02/18/0324238from=rss
description: SumDog writes "The UK is known for many things, great food,
a wonderful climate and beautiful women. However, according to a story on the
Guardian, a new study puts the UK ahead in one more category: it leads the
world in TV piracy, accounting for 38.4% of the world's TV downloads, with
Australia coming in second at 15.6% and the US in third at a pitiful 7.3%"
date: 2005-02-18T09:31:00+00:00
author: CowboyNeal
title: Skype-Ready Phones From Motorola
link: http://slashdot.org/article.pl?sid=05/02/18/0314225from=rss
description: Hack Jandy writes "Seamlessly integrating VoIP and GSM might
not be a fantasy after all, as Motorola announced their decision to build cell
phones and handsets that have Skype Internet Telephony integrated into the devices.
Obviously, one could use Skype for outgoing calls near wi-fi hotspots (essentially
free) but default on GSM for outgoing calls in areas that lack coverage."
date: 2005-02-18T06:09:00+00:00
author: CowboyNeal
title: London Nuke Plant Loses 30 Kilos of Plutonium
link: http://science.slashdot.org/article.pl?sid=05/02/18/0027246from=rss
...........

Our first function will extract all item elements.
To do this we create a function getElementsByTagName/2 which
takes the XML parse tree and the Tag that we want to find.

The RSS aggregator.

To go from here to a RSS aggregator is easy. You just have to extend the code above with the functionality to retreive info from several RSS feeds. You may also want to present the info in some other format, e.g HTML via a Yaws page. This however, is left as an exercise for the reader to do.