Getting data from XML feeds using PHP

Introduction

In a previous article, I introduced a set of simple functions that grab data from XML files. The functions are a little rough and ready, but with well-structured XML data formats they work very well. This article offers an example of how to use those functions to grab the latest headlines from a real-world RSS feed and then format them into a "latest news" box for your website.

The desired result

Here is the result of fetching an XML feed, extracting the desired data from it, and formatting that data into a block of HTML. Using CSS, you can alter the appearance of the news box radically, choosing colours, sizes, positions and fonts to suit the situation.

Have your say: Best prime lens of 2016
It's a great time to be a photographer. Based on reader interest and preferences, we've rounded up a total of twelve prime lenses that were released in 2016 for your consideration in...[12:00, 9th December GMT]

Have Your Say: Best Zoom Lens of 2016
The end of the year is almost upon us, and as usual, 2016 saw plenty of new lenses released, including several excellent zooms. We've used a lot of them and tested some, but we...[12:00, 9th December GMT]

The EOS M5 is Canon's current enthusiast-grade mirrorless camera. With a 24.2MP APS-C sensor, Dual Pixel autofocus and a pleasant-to-use interface, the M5 is the most competitive EF-M camera Canon has released.
We've put...[23:11, 8th December GMT]

For those wanting to step up from entry-level to midrange ILCs, there are many things to consider, including the choice between a DSLR or mirrorless camera, what sensor size suits you best, how important video is to you, and of...[22:36, 8th December GMT]

Feed fetched 2016-12-09 14:59:31 UTC

Step one: Get the RSS feed

Before you create a script that fetches an XML feed from someone else's site, make sure to check that you have permission to do so. Look for a usage policy in the feeds section of the site, or email the site owner to make sure they're happy with what you intend to do.

For this article, I'm using the latest news feed from Digital Photography Review, one of my favourite sites. The XML feeds at dpreview.com are in the RSS 2.0 format, but the techniques used in this article could be applied to most XML data formats.

Once you have the URL of the XML feed that you are going to use, you need to have PHP load the contents of the feed into a string variable. Using file_get_contents, you could fetch the XML file like so:

$xml = file_get_contents('http://www.dpreview.com/feeds/news.xml');

However, this requires that PHP is setup with allow_url_fopen set to true. Not all web hosts enable this setting, for security reasons. So another way to fetch the XML file into a string is by using the cURL functions (if they are installed on your PHP setup) like this:

Either method should give you a string called $xml that contains the contents of the entire XML feed.

Step two: Extract data from the feed

Most XML data files contain a large amount of content. For instance, the dpreview.com news feed contains channel (feed) title and image information, a couple of dozen news items, each with a title, description, image for the item, link to visitor comments, and the publication date. To keep things simple, I'm only going to extract the channel title and URL, and the title, description and date of each news item.

// Include the handy XML data extraction functions.
include 'xml_regex.php';
// An RSS 2.0 feed must have a channel title, and it will
// come before the news items. So it's safe to grab the
// first title element and assume that it's the channel
// title.$channel_title = value_in('title', $xml);
// An RSS 2.0 feed must also have a link element that
// points to the site that the feed came from.$channel_link = value_in('link', $xml);

Next we want to build an array that contains individual news item elements. This can be done using the element_set function.

Once we've got the array of item elements, we can iterate through the array, one item at a time, and extract just the data that we're interested in from each news item: the URL, title, description and date. Then we can store that data in a new array called $item_array.

Now we have $item_array, which is an array that contains news items in the form of associative arrays, so that the title of the first news item should be found in $item_array[0]['title'] and the title of the second news item should be found in $item_array[1]['title'] and so on.

Step three: Markup the data with HTML

With all the data we want stored in one handy array, we can iterate through the array and markup the data with HTML.

The above code creates the HTML for the first five news items, plus the channel title, and wraps it all in a div element with the class "rss_feed_headlines" so that CSS stylesheets can target the news box specifically.

Step four: Consider security implications of external data

I trust the people at dpreview.com not to send anything malicious to me via their RSS feeds. But suppose their site was hacked or infected by malware. Then they wouldn't be in control of what their RSS feeds contained. And my script would be fetching a corrupted RSS feed and displaying it on my site, which could allow for cross-site scripting exploits to run riot.

So you need to write code as though the RSS feed is suspect, even if you trust the site that you are getting it from. This means doing what is necessary to avoid any third-party HTML making it into your page. The number of cross-site scripting exploits is high, so it can be difficult to know how secure your code is.

To reduce the number of exploits that can survive, my code calls a function named make_safe on everything that is going to be output in HTML.

This function removes CDATA sections (see the note below), then calls PHP's strip_tags function to remove HTML markup, then tries to convert any remaining dangerous characters into HTML character entities (PHP 5 is better equipped to do this than PHP 4). However, if you see any holes still open to exploitation, tell me about it.

Embedded markup and CDATA sections

Some RSS feeds embed HTML markup in their title and description elements using HTML entities. These are harmless, but they will cause the news item to appear on your page surrounded by HTML tags. There are various ways to remove such unwanted markup, but each feed will require its own adjustments, so I don't offer suggestions here. [See February 2012 update below.]

Another issue is that some RSS feeds contain CDATA sections to hide non-text elements inside RSS elements, such as HTML markup. To get rid of these CDATA sections, my code deletes everything in them with the following code:

$string = preg_replace('#<!\[CDATA\[.*?\]\]>#s', '', $string);

The RSS feed for dpreview.com only uses CDATA sections to store img elements inside description elements, but some RSS feeds seem to wrap everything in CDATA sections, even plain text. So, as with removing unwanted markup tags, you may need to craft a solution customised to the content of the feed you are using.

Other XML feed formats

The code in this guide has been based on parsing an RSS 2.0 feed. Some websites only offer Atom feeds for syndication. The Atom standard uses empty elements for some information, such as its link element. In such a case, you can use the element_attributes function to extract data from XML elements, and then the rest of the code suggested above can remain very similar.

The above methods aren't limited to syndication feeds. You can use the functions to parse data out of XML produced by website APIs. For instance, the Amazon Associates system returns results in XML, and you can use these functions to parse the data you want out of the XML.

Updates

31 January, 2010: For anyone using PHP version 5.2.3 or later, I recommend you take a look at my article about reading XML feeds using SimpleXML, which shows you an easy way of constructing an object tree full of data from an XML document.

25 February, 2012: The dpreview.com feed has (for some time now) been placing HTML markup inside its description elements, which is why you might have seen img and p elements in the text of the news feed display at the top of this page recently. To remove these I've modified the make_safe function so that after the first call to strip_tags the resulting string is fed to html_entity_decode to convert entities back to raw characters, and then strip_tags is called again to remove any HTML tags which result. This seems to do the trick, but I've not got a full PHP development platform setup on my machine at the moment, so I've not been able to do any testing. Also, you need at least PHP 4.3.0 to use the html_entity_decode function so this won't suit some users (though as PHP 4 is no longer in support you really should have moved to PHP 5 by now).