However, I determined that it would need to prove itself in some comparison testing with other GPS units before it could earn the right to be considered a quality geocaching tool.

Facing the iPad/Bad Elf combination in this GPS Royal Rumble is the little old yellow legend of geocaching, the Garmin Etrex, my brand spanking new Garmin Oregon 450 (keep an eye out for my review on that) and my favourite geocaching tool – the Blackberry Bold 9000 with Cachesense.

Blackberry Bold 9000

Garmin Etrex

Garmin Oregon 450

These units have gone head to head in a series of three tests – Satellite Acquisition, Geocache Accuracy and Waypoint Marking.

Before you delve into the results, I will acknowledge a numbers of things:

I’m not a professional GPS tester.

The tests are designed to represent geocaching ability, rather than general accuracy.

There are undoubtedly a huge number of variables within each test that can impact the results – I will acknowledge some of these where I think they are relevant.

My sample sizes in each test are not huge and thus the results are not likely to be statistically significant … they are interesting none-the-less.

Satellite Acquisition Testing

This test was to ascertain the length of time each unit took to go from a cold start to being ready to navigate. Three different locations were chosen with three different types of environments:

An large area of clear ground away from buildings and trees with lots of satellite-filled sky above.

An area amongst high rise buildings with a small amount of tree cover.

An area of heavy tree cover with little to no sky visibility.

All areas were tested on different days and from a cold start (the GPS had not been started that day).

The raw data from those tests is below, with most figures denoting seconds.

iPad/Bad Elf

Garmin Oregon 450

Blackberry Bold 9000

Garmin Etrex

Clear, no cover

37.28

12.28

16.27

39.59

High rise buildings and some trees

40.67

14.60

17.52

59.38

Heavy tree cover

1.35.18

20.15

59.24

3.55.04

Average

57.71

15.67

31.01

1.51.33

There is a very clear winner here – the Garmin Oregon 450. It was clearly faster in all three locations and its averaged result was a whopping 15 seconds than the next best.

Proudly, the Blackberry Bold 9000 produced a solid second. I thought its acquisition time for the clear and high-rise sections was particularly good.

The iPad/Bad Elf comes in a very respectable third here. I was pleased to see that the difference in acquisition between the clear and high-rise areas was minimal. It lead me to ponder whether a good chunk of the acquisition time was the unit starting and communicating with the iPad.

I was also very pleased with the heavy tree cover result for the iPad/Bad Elf. Not only because 1.35.18 is not long to wait in heavy cover but because it showed it had at least 10 metres accuracy right from acquisition time (and you’ll recall that iOs seems to prevent any accuracy better than that from being displayed). So from the time it picked up it was ready to go.

In last place was the little yellow legend, the Etrex. Respectable times in general but I nearly gave up on it under heavy tree cover. Four minutes is quite a long time to wait! And unlike the iPad/Bad Elf, its satellite accuracy was pretty poor.

Geocache Accuracy Testing

For me, the Geocache Accuracy tests and results were the most interesting. For this I selected four geocaches of varying age and locations to see how far away each GPS unit said ground zero was at the cache site.

I see this is an important test because it gives an indication of how close this GPS is going to get you to geocaches.

For this test there are a huge number of variables that can impact the results and these include, but are not limited to, the following:

‘Accuracy’ of the GPS when placing – we know different GPS units tend to produce different results and therefore you are more likely to get closer if you are using the same type of GPS (or so I’ve been told).

The age of the cache – obviously caches placed a long time ago would have been done with older, less accurate units. I have accounted for this by including a range of cache ages.

We don’t know what units the caches were placed with.

Four caches is hardly a statistically significant number but its enough that we do get some patterns forming.

The caches selected have the following characteristics:

The Stadium: A ‘newer’ cache hidden in bushes in a sportsground.

Not Quite Everest: A cache of moderate age hidden is a relatively open area.

Nu Nutcase: A cache of moderate age hidden in a very open area.

Botanical Gardens: A very old cache hidden in a bush area with fairly significant tree cover.

All of the hides were placed by cachers who have considerable experience (and as such you would assume they were placed properly with quality readings etc).

Before reading the results below, it is worth noting that they are not necessarily an indication of the true accuracy of a GPS unit, rather an indication of its accuracy in relation to a geocache site.

The figures below are in metres and represent how far away each unit said ground zero was at the cache site, along with the direction they indicated. You can use a conversion calculator if metric is not your thing.

The results here are fascinating. The Blackberry comes in first place on all four caches. Sweet, sweet proof of what I have known for a number of years now – that the Blackberry with Cachesense is an awesome caching tool. An average of 3.5m out is not bad and a good indication that the “where you are search engine” tag line is so true.

But in the context of this testing the most significant result is that in second place is … the iPad/Bad Elf! Narrowly edging out the Oregon overall by 0.5m. They beat each other on two caches each, so some may want to argue it was a draw. 😉

What the result really shows is that the iPad/Bad Elf combo is a quality caching tool that can foot it on Geocache Accuracy with the best.

And, as you would be expecting by now, the little yellow legend came in last place.

Waypoint Marking Testing

The last set of tests involved marking a waypoint in the same place with each of the units and then comparing the results.

The point of this test is to not only show that different units produce different results but also potentially begin to demonstrate that some units can cluster together.

With limited time I was forced to only mark one set of waypoints, where I would have liked to have produced at least three sets. So the results shown are really only one sample and should be treated as such.

To use the grid select the unit you want down the left-hand side. The co-ordinates from its waypoint are along the row, as the distances from the other units.

iPad/Bad Elf

Garmin Oregon

Blackberry

Garmin Etrex

iPad/Bad Elf

S 45 51.988

E 170 31.442

7.4m

2.6m

4.5m

Garmin Oregon 450

7.4m

S 45 51.992

E 170 31.442

7.9m

11.2m

Blackberry Bold 9000

2.6m

7.9m

S 45 51.988

E 170 31.444

4.5m

Garmin Etrex

4.5m

11.2m

4.5m

S 45 51.986

E 170 31.442

It is difficult to draw any real conclusions from this, especially because it is only one sample. Of interest in the results is that the Oregon produced a waypoint that was further in distance than any of the others. The initial impression would be that the Oregon did not perform well here but that is actually incorrect. If we place the points into Google Earth and look at them on street view, it shows a slightly different picture.

Putting aside any issues regarding whether co-ordinates in Google Earth provide an accurate representation of their location what this image alerted me to was that the Oregon and Blackberry co-ordinates were likely to be the closest to where I mark the waypoints. I estimate that to be about where the front of the blue car in the image is. Considering this the likely outcome was the Oregon narrowly wins over the Blackberry with the iPad/Bad Elf a couple of metres away. Note that the Etrex is halfway across the world.

Other Tests

Whilst they weren’t available for this testing I’ve also been able to plug the Bad Elf into an iPhone 3 and an iPhone 4.

The owner of the iPhone 3 had never received horizontal accuracy of better than 17m using the Geocaching App, but the Bad Elf instantly improved the accuracy displayed to 10m (and we know it was probably better than that).

The iPhone 4, my wife’s pride and joy, gave me a little more play time. Using an app called GPS Info, I first tested the GPS accuracy without the Bad Elf. The horizontal accuracy was shown as 10m whilst the vertical accuracy was 19m. After plugging the Bad Elf in, the vertical instantly improved to 10m, but what happened with the horizontal was most interesting. The horizontal, for just a few moments, displayed 5m accuracy with the Bad Elf. Then, as if iOS had said “No, no, no it must be 10”, the accuracy reverted back to 10m.

The end result of all this is that it appears the iPhone 3 and iPhone 4 both benefit from use of the Bad Elf. If my tests above are anything to go by, it suggests they would make very good caching tools indeed.

Bad Elf Conclusions

Okay, I’ve tested it in the field, I’ve mucked around with apps and I’ve compared it was other GPS units and here’s what I think …

The Bad Elf is an excellent GPS accessory – it certainly does improve iOS GPS accuracy and easily provides GPS to an iPad. In comparison with other units, it performs to a high degree and one that is quite suitable for geocaching.

An iPad is not the most comfortable geocaching tool – it’s big and bulky and outside of an Otterbox protective case, you worry about dropping it. But I can see the iPad + Bad Elf being excellent on a caching road trip as the in-car unit.

iPhone + Bad Elf = Gold – this combo would provide you with a quality caching tool that can pretty much do it all.

Bad Elf is definitely value for money – at US$99 the Bad Elf is similar in price to the older Etrex models. What the results above show is that it performs more like an Oregon. So it you have an iPhone and have been pondering the purchase of a new GPS unit, you should seriously consider getting a Bad Elf instead. At a fraction of the cost with similar results, how can you argue with that?

The Geocaching Apps still have a way to go – especially when you compare them with the likes of Cachesense for Blackberry (and hopefully soon Android). My one complaint is that these apps do not seem as advanced and as a result it seems to reduce the stability of the navigation a little. The compass can spin a little too easily, even when you are moving at a good speed. And not being able to see the true accuracy is just plain annoying. Fingers crossed both of these issues will be addressed before too long.

Overall – I love the Bad Elf and am super stoked I have one. Despite the fact that I will likely continue with my Blackberry and Oregon, predominantly because the iPad is not robust enough, I expect to keep enjoying the iPad/Bad Elf combo. It has been very useful so far and for way more than just geocaching. Overall I’d give the Bad Elf accessory a 5 out of 5 (but I’d only rate the iOS and apps a 2 out of 5 at the moment).

I know that kjwx has a Bad Elf coming to use with her iPad and iPhone 4 and it will be interesting to see what her experience is…

Clan Riffster

Another set of tests I would recommend is direct comparisons between an iPhone 3, iPhone 4 and iPad using just the included hardware, Vs. those same three devices using the Bad Elf.

My favorite test revolves around returning to a known marked waypoint.

I did a comparison between several commercial handheld units similar to one of the tests you conducted, but with a slight twist. First, I found three suitable locations, a tree stump in a field with an open sky, a manhole cover next to a road with moderate tree cover and an orienteering marker under heavy tree cover.

My first task was to go to each location and acquire two sets of coords with each device. The first set was a snap reading, taken as soon as I arrived. The second set were after letting the devices settle/average. Then I returned three times the next day, morning noon and dusk, (to account for different satellite configurations), and see what the distance was from ground zero with the devices held directly over the spot where I acquired the coords. Again, I grabbed two sets of coords per device. One as soon as I got there and the other after the device settled.

To my way of thinking, there really is no good way to test accuracy when pitting your GPSr against another person’s GPSr. As you noted above, the variables are too broad to give results which could be called accurate. That’s why I like to test a particular GPSr against itself, to see how well it does returning to a known spot.

Great minds think alike, Sean … Cumbyrocks and I both have plans to conduct such a trial, though I have to wait a few more days for my own Bad Elf to arrive Down Under. I like your idea of repeatedly checking accuracy at a known waypoint so we’ll be sure to add that test to our methodology.

Jim

Just wondering if you have any plans to also check out the Spot Connect GPS antennae? I like how it communicates via bluetooth and not the 30 pin connector, allowing a person to encase their ipad in a robust weatherproof case, but will have the antennae open to the sky. With the Bad Elf the closest you could get to that would be a 30 pin extension cable, but then you’d be messing with the seal of whatever case you put an ipad in.