Tuesday, July 26, 2005

I noticed that Google Earth ships with something called GPSBabel (and my old friend expat.dll). Now I knew almost nothing about the world of GPS devices but I soon found that there are (too) many GPS file formats in the world. GPX seems to have become the de-facto XML standard but because of the wide variety of GPS hardware there are still plenty of different plain text file formats kicking about. GPSBabel is GPS format conversion software. I noticed that Google Earth can load GPX files! I found that using free GPS conversion utilities like GPS utility, G7toWin and GPSBabel I can pretty much convert anything I find on the net into a format that I can import in Google Earth (the freeware version!).

Secondly, I thought I could write an XSL to convert GPX to KML and vice versa but I found to my delight that GPSBabel has provisional support for converting GPX to KML, so I can leave the XSL stylesheet to someone who really needs to do it.

All this is good so far but what if I want to put the route of a walk on my website using Google Maps API. How difficult is it to convert KML (or GPX for that matter) into the Google Maps API. I wrote two stylesheets that generate the Google Maps JavaScript that make a map containing a "GPolyline" of a route. I used a GPS text file of walk around the Lake District in Cumbria (in England). Surprisingly it is *much* easier to transform GPX to Google Maps than the KML format. My conclusion would be if you're veering towards using Google Earth professionally then don't throw away your GPX files until someone writes an XSL to convert them back from KML format (you need an escape strategy).

Monday, July 25, 2005

I have managed to combine using CASFilter and SecurityFilter. This means I can use CAS for authentication and Tomcat Realms (e.g JDBCRealm, DataSourceRealm, JNDIRealm) for the actual role authorization (isUserInRole).

The KMZ format looks to be simply a PKZipped version of the KML file. Although rather interestingly there is scope to add images and perhaps other media (e.g. 3d models, video and audio) into the zip file.

It seems that at present the main body of Google Earth developers are using PHP but as you can see from my recent posts I have been having some success developing for it with JSP technologies.

First, I tried to load a GZipped file into Google Earth but that didn't appear to work. It looks that Google Earth currently only supports "PKZIP" style files at present.

I thought to myself that I could create a Servlet Filter to dynamically create PKZIP files. I found an old article entitled Two Servlet Filters Every Web Application Should Have (circa 2003, virtually Jurassic!) on O'Reilly's OnJava that describes how to create a GZIP compression servlet filter to automatically compress the output of JSPs. I need to produce something very similar but this time with the emphasis on PKZIP.

It took a little longer that I expected to get my KMZFilter working. It seems that "all modern browsers" are assumed to accept the gzip encoding. Therefore there no longer seems to much point in searching for "gzip" in the accept-encoding request header as most browsers don't bother to send that header anymore. Additionally it isn't quite as straight forward as changing all the java.util.zip.GZIPOutputStream references in the GZIPFilter to java.util.zip.ZipOutputStream either.

The KMZ servlet filter version I produced does no yet support zipping up multiple files. To support multiple files I'd need to work out from the requested filename (e.g. index.kmz) the file list that I'd need to include in the zip file, perhaps an XML properties file would do the trick (actually that would be quite an elegant solution).

My web.xml contains the following, note that I have mapped the KMZ extension to JSP so that I can carry on writing JSP to do the actual KML file creation:

Kudos to Jayson Falkner as I stole most of the code from his GZIPFilter but you are very welcome to have the source code filter for the KMZFilter I produced. I make no guarantees about the filters reliability, actually that probably goes for almost everything I produce (I will not be a hostage to responsibility)!!

[Update]Inspired by the compression filter example that comes with Tomcat I have improved my filter and it works much better now.

$CATALINA_HOME/webapps/examples/WEB-INF/classes/compressionFilters

This was not helped by the Servlet Filters responses of <html><body></body></html> rather than an error I could actually do something with! If I were being professional about it I'd also add some code to make sure, even when responding with a servlet exception that the response from this filter was always KML/KMZ.

Saturday, July 23, 2005

Find only Google EarthKML files by using the well known "filetype:" Google hack. There are around 5290 KML files indexed by Google at the time of writing this. Since KMZ files are binary it doesn't seem worth searching for those at the moment.

Friday, July 22, 2005

In my last blog entry, I showed how you could get a web service and Google Earth client to communicate using a KML file dynamically generated by JSP. Now I'm going to extend this approach so that a list of blog in the immediate vicinity of a location are displayed inside the Google Earth client by making use of the GeoURL RSS feed.

Essentially this is quite easy because in the last entry we already produced a script that could calculate the centre co-ordinates of a Google Earth map from details passed to it from the client application. Since we have the central map co-ordinates we can query the GeoURL feed for blogs in that vicinity. The final stage is to convert the RDF feed returned into the KML format that the Google Earth client requires.

In my XSLT I borrow a few tricks from one of my *favourite* XSLT related articles Never Mind the Namespaces: An XSLT RSS Client (which is incidentally very useful if you want to create an XSLT that can transform multiple RSS formats). There are a couple of namespaces knocking about in the RDF feed that GeoURL returns and for my purposes I'd be happier ignoring them. So here is the XSL stylesheet which will transform the GeoURL RDF into KML format.

Google Earth is cool despite being proprietary software which is not usually my thing. A few weeks ago I did some experiments with the Google Maps API and this was quite successful. I've just discovered that there is an "API" of sorts for the Google Earth client and thought I'd give that a whirl too. Google Earth's underlying communication format is called KML (an XML format). See:

This essentially informs the Google Earth client of the details of your web service, when and how it should be called. Once you've written your dynamic script (as below) then all you need do is load it in your web browser [e.g http://localhost/GoogleEarth/test.kml]. Once installed Google Earth should pickup KML files automatically.

You'll see in the above document that I've asked it to retrieve a JSP page from my local server. This is the JSP page which will dynamically create KML documents and will be called periodically (at an interval defined in the previous document) by the Google Earth Client:

I've used the Jakarta String tag library to split the BBOX parameter by the comma separator but apart from that this is JSTL. This JSP page is called after the "camera" comes to a stop in the Google Earth client. So you are presented with a new marker after every pan around the globe comes to a halt. In my next entry I plan to describe how I managed to extend this method to combine Google Earth with GeoURL feed data.

[Note] In my case I ran these scripts on Tomcat 5, depending on your server you may or may not need the addition "_rt" on the end of the JSTL tag library references.

Tuesday, July 19, 2005

IBM alphaWorks aDesigner is described as "A disability simulator that helps Web designers ensure that their pages are accessible and usable by the visually impaired." It is a desktop client based on Java and Eclipse technology.

What is particularly nice about it is that it allows me conduct accessibility tests on web applications that require authentication; where external online accessibility validators like WebXACT (previously called Bobby) and Cynthia Says wouldn't be able to access. To be fair I believe you can purchase a version of Bobby that can be configured to work with authentication protected web pages ( there is also a desktop version of Cynthia Says which might possibly also work). Although I'm easily discouraged from using any software that I have to pay for.

Unlike with the online versions of Cynthia Says and WebXACT, with aDesigner I was able to login to my instance of uPortal and test the pages directly. Prior to this tool I had to conduct any accessibility testing on static dumps of the portal pages.

Also as far as I can make out it is free for Academic use. The only possible downside I can see so far is that you need to register with the IBM site to download it.

I discovered whilst browsing through Google Print that Shawn Bayern (in his excellent book JSTL in Action) described using an XMLFilter with JSTLs <x:parse>. What an XMLFilter allows you to do is discard the parts of an XML Document which you don't need before further processing. The XMLFilter is SAX rather than DOM based which apparently means that it can be more efficient in terms of the processing time and the memory it uses. The DOM method would need to build the entire document in memory before any processing can occur, whereas SAX works more like a parser reading only those parts of the XML document that it needs to.

In Shawn's example he created a taglib called Spath that is used to produce an XMLFilter via a limited subset of XPath. Part of SPath is generated with JavaCC. This all looked a bit complicated for me to use for my own purposes. I then discovered that it is actually quite simple to create an XMLFilter from an XSL transformation document.

For small XML documents you are unlikely to gain anything from adding an XMLFilter but if you are dealing with XML documents of a reasonable size then an additional filter can make quite a difference.

As an example I obtained a copy of Hamlet in XML format (I removed the DTD reference from it as it was causing trouble). For the filtered example I discarded all but the first scene of the first act, in the unfiltered example I processed the whole play (apologies for the scriptlets).

The result is the filtered document processing looks like it is faster than the unfiltered document processing. Granted this is not a very scientific test and I'm not claiming that it is the most efficient use of either SAX or DOM techniques. It does however highlight a relatively easy way to make use of an interesting and little known feature of JSTL.

Monday, July 04, 2005

Previously, I showed how to display the blogs in your immediate vicinity using GeoURL and Google Maps with the Official Google Maps API. I received a comment following this from Ask Bjørn Hansen who runs the GeoURL site. Ask suggested that it would be really good if you could make it so that as you scroll around the map it automatically updates the display showing the blogs in that vicinity. I took this as a bit of a challenge (I occasionally like the odd challenge) and I like to play with AJAXy stuff at the moment with it being flavour of the month.

It is easy using the Google Map API to add an event listener to monitor map scrolling events and read off the longitude and latitude readings. There is even an example of the code needed to do this in the API documentation. After looking through some of the GeoURL source code I discovered that I could receive a GeoURL RSS feed based on a longitude/latitude pair. My first attempt fetched new RSS feeds after every "moveend" event, this initially appeared to be working until the browser slowly ground to a halt. The problem was that I had been adding more and more "markers", those little red icons, until it just became too much for the browser to handle.

The API documentation gives three methods for dealing with markers these are referred to as overlays in the API documentation (as they can be a icon marker or a series of points joined together). These are:

addOverlay(overlay) # Adds the given overlay object (e.g., GMarker or GPolyline) to the map

removeOverlay(overlay) # Removes the given overlay object from the map

clearOverlays()

So therefore, I can add an overlay, remove an overlay (if I had a reference to it) or clear all the overlays on the map.

I modified my code to add the clearOverlays() command before adding any new markers and this made the code work better. The problem now was that the markers would blink on and off as they were removed and re-added after every new RSS feed load. This was not very satisfactory.

What I needed to do was this, after a "moveend" event:

Remove overlays that were not now inside the bounds of the visible map (called viewport in the API documentation)

Only add overlays if they would be visible within the bounds of the visible map

Now there didn't appear to be a way to iterate through the existing overlays on a map page, at least not in the official API documentation. I can forgive this as Google Maps is only in Beta and I put it down to an accidental omission in their documentation. After looking at a Google Maps Hacking page I saw a reference to overlays being a property of the map object. I thought that must be the array containing all the overlays in the map. By guesswork and pure dumb luck I was able then to access the longitude and latitude values of each overlay using:

map.overlays[0].point.xmap.overlays[0].point.y

Now I can access the overlay objects, their associate co-ordinates and I can delete the overlays using removeOverlay(overlay) method. I can now remove the overlays that would be outside of the visible map.

Hey presto, it works...it only took about a half day of head scratching. The following is a link to the live example, it works but I'm sure it could probably be improved further.

You can of course copy the static HTML from the above page, I have opted for a Creative Commons license for the code I produce on this site.

It required a modified JSP page to proxy the RSS feed so that the AJAX code could read the XML from a local source. This jsp will fetch a GeoURL RSS feed if called with a reference to an URL or a longitude and latitude pair (For reference: I currently use JSP on Tomcat 4 and I believe it would need to be modified to use the JSTL core_rt taglib on Tomcat 5).

Sunday, July 03, 2005

This morning I didn't really know what a GeoURL was, I've seen the button badge on a few blogs but I didn't get as far as clicking on them. It turns out that getting a "GeoURL" is really easy, all you need is the longitude and latitude of the blogs location, you can get this from multimap and the like, and you then just insert a couple of metatags in the head of your webpage and register with GeoURL (they even give you the HTML for the button badges). In my case the metatags are:

Now as anybody who knows me will tell you, I am a good boy ;-> so when I saw that those clever chaps at Google had realised an official Google Maps API I just had to see what I could do with it. I thought why not try and use it to emulate Leigh's clever little application and blimey, I only went and got it working!!!!

Some interesting points I found along the way, the Google API requires you to sign up for an API key with your domain name in order to use it. It looks to me that this API key is actually verified in the client side JavaScript because in order to get my Tomcat server working I had to specify the domain localhost:8080 (API key for localhost:8080 shown in the example code below, I hope I'm not breaking any rules). The live example uses another API key obtained specifically for use on this blogs server (including the application directory). Out of interest, I did try to look at the Google JavaScript but anyone would think they were trying to obfuscate it or something (strange for a company that are starting to embrace the Open Source culture, see: Google code).

The official API uses AJAX technology to obtain the XML and process it. Due to security concerns the XML that the JavaScript accesses must be located on the same host as the JavaScript page. In order to obtain the GeoURL RSS feed that I need to use I needed another JSP page that downloads this feed on my behalf so that it appears local for the JavaScript's benefit.

I use Firefox browser but I recognise that most people would want this to work in IE (I recognise it but still don't understand why!!). The GeoURL RSS stores two of its key elements (longitude and latitude) with a special GeoURL XML namespace, as you would expect as these elements are currently not part of the RSS specification. I found that I had to do some browser detection in my JavaScript in order to get the namespace stuff working. This is because IE does not support the getElementsTagnameNS method and Firefox doesn't support the way IE supports obtaining tags with namespaces (no surprise there as the IE method looks just plain wrong).

One final point is I have learnt how to spell longitude, I’d always previously thought the word was longtitude (you live and learn!).