Living life one line of code at a time – by Andy Gup

Archive for the ‘GPS’ Category

If you are headed to OSCON, swing by my session on Mastering Android Geolocation. Itâ€™s a deep dive into the Android SDKs android.location package. If youâ€™ve ever wanted to learn about the fundamentalâ€™s of the SDKs Geolocation capabilities then this is a must attend session. It will also give you a strong foundation to understand the underlying capabilities of the new Google Play Services SDK that includes Fused Location, Activity Recognition, and Geofencing APIs.

The presentation includes digging into the capabilities of my open source GPS Testing toolÂ that lets you easily test different aspects of the Geolocation capabilities.

In part 1, we looked at the six types of location data available through the Android native SDK and talked about how to work with cached location data. As others have also discovered, there is a ton of information to say on this topic and way more than you could pile into a single blog post. So, this post focuses on working with real-time locations. Once you activate the location listeners they will start sending you information and knowing how to handle that data is what this post is about.

The most important take away I hope to leave you with is take time to understand your accuracy requirements and your userâ€™s basic geographic behavior patterns.Â I’veÂ tried to liberally sprinkle example use cases to help illustrate some of the concepts.

Time and distance properties. The first thing youâ€™ll notice when you start building location aware Android apps is you can set time and distance properties that affect how often the device will request a location update. This is accomplished using the overloaded LocationManager.requestLocationUpdates() method. These properties adjust how aggressively the device will request location updates by minimum time elapsed and minimum distance traveled. How you adjust these settings significantly affects the battery life. The equation for battery life isÂ straightforwardÂ and simple: the shorter the minimum time interval the faster the battery will be drained.

In the code snippet below, the time and distance properties are both set to zero, which is the most aggressive setting. Â Youâ€™ll get a location update if any change in location is detected at any time. This setting can result in your app accessing the GPS provider as often as 50 â€“ 60 times or more per minute.

Iâ€™ve experimented with this setting by leaving it running, and it drained a fully charge Samsung Galaxy SIII in less than 3 hours. The phone also got very hot to the touch. It takes considerable power to run the current generation of GPS chips. Even using network location requires that the phone use its radio to make an internet request to the location processing server.Â So, carefully evaluate your use cases and be conservative about the time and distance properties that are needed.

Except for high performance situations, it will be rare when you need location access every second at a zero distance delta. You can also adjust these properties if your app detects a change in the usage pattern. For example, if the user slows down for a period of time then you might restart the listener using a longer time interval. The listener does not have to stay static and be set only once. Hopefully this is your â€śah-hah!â€ť moment. In fact I encourage you to make dynamic adjustments to the intervals as your usage scenarios change. The GPSTester tool allows you to easily experiment with different configurations.

Here are some use case examples:

Real-time tracking app for use when walking, running, biking or driving. Â As expected, this app will require the most aggressive settings for time and distance.

Delivery vehicle tracking app. The delivery vehicle will be starting and stopping all day long. Minimal ability to plug phone into charger. This app should dynamically adjust its time and distance settings.

App only needs a one-time snapshot location to find nearby coffee shops. This requires that the app run until a minimum accuracy value is received and then it can turn off the location listeners. Other examples are placing a geotagging a photo, tweet or Facebook posting. You could also try your luck using the overloaded Location.requestSingleUpdate() method.

And, hereâ€™s a very basic set of technical requirements for dynamically changing requestLocationUpdates() settings:

Start application using minTime = 0 and minDistance = 0. Use an aggressive setting to get accurate location as quickly as possible.

Once accuracy is less than 50 meters and speed less than 45 mph set minTime = 5000 and minDistance = 25.

Time to 1stÂ real-time location.Â It can take up to several minutes to get the first GPS or Network location result. Use effective notifications to let your users know there will be a delay in getting this information. For example, many Android mapping apps use an accuracy circle around the current location indicator to give a sense of accuracy level.Â

As you can see in the GPS ProviderÂ screenshot taken from the GPS Tester app, using a warm GPS it took ~9 secs to get the first GPS result from the device and then 1450ms to get the next update for an accuracy of 3.0 meters. Â By warm Iâ€™m referring to a device where the GPS and Network location have been accessed recently.

How long it takes to start getting accurate results depends on many different factors of which you, as the developer, will have very little control over but youâ€™ll need to plan for accordingly.Â As discussed in Part 1, many developers lean heavily on the cached locations to try and work around these types of delays. Yet, depending on your use cases and the end user personas these results can either be very useful or fantastically inaccurate.

According to Google user interface responsiveness guidelines 100 to 200 milliseconds is where users will start to perceive slowness and we are talking about 9 whole seconds here that the user had to wait. Of course 9 seconds is â€śfastâ€ť for a GPS acquisition. And, you can typically expect much longer time frames, especially on a cold device, and until the accuracy drops down to within a few hundred meters or less.

Iâ€™ve seen differences in acquisition times between two phones of the same exact model and same settings even though they were placed right next to each other. Other factors that affect acquisition times can be as simple as where you placed the phone in the car such as down by the gear shift between the seats or up on the dashboard. Or the user could be standing just inside aÂ restaurant under a metal awning because itâ€™s raining outside. Â The list goes on and on. Cement, bricks, metal, car bodies, heavy foliage and buildings are some other examples of things that can interfere with GPS signal accuracy and the time to first acquisition. Like the Boy Scouts say, â€śBe Preparedâ€ť and be kind to your users by using user interface notifications to let them know of location acquisition delays.

Streaming real-time locations. Once the device starts providing real-time locations, my suggestion is to check the accuracy of each result and consider rejecting those greater than a certain amount that are based on your requirements. The longer the location provider runs, and if the device has an unobstructed views of the sky and good cellular connection, then typically the accuracy will improve up to a certain point and then level off, and then it will fluctuate. Â Hereâ€™s a pseudo-code snippet showing how to check the accuracy of each GPS location result:

Here are some rough examples of accuracy thresholds I used for a project last year. Your requirements may vary as to how these different thresholds will affect the behavior of your application; these were examples that required geocoding that converted the current location to an approximate address. Depending on the result the application gave different feedback to the user:

City > 500 meters and <= 2000 meters (ask user to verify city name from a list)

County > 2000 meters (prompt for manual location input)

Take into account your own unique use cases. You might completely reject any accuracy value greater than 100 meters (328 ft) if your app simply helps people find open parking lots at NFL games. You could have an app that returns a list of Dentist offices within a 5 mile (8000m) radius. Or a weather app could only need to know approximately what city you are in. These are just ideas to help get you thinking.

Not all real-time location data is alike. Â Continuing on the theme of consuming real-time location, letâ€™s dig into a few more examples of why it should be looked at closely. Hereâ€™s one thing that I keep forgetting even though Iâ€™ve blogged about it: you can get null location values and if not properly handled they will crash the app. Thatâ€™s an easy one to forget until your users start to report random application crashes.

You can also look for what Iâ€™ll call spurious results. These are results that are way outside what you might consider a running average. Check the distance between the last location result and the current location result using the Location.distanceBetween() method. If the distance and speed required to cover that distance is significantly greater than the running average, then you can reject that result.

I also want to mention that simply holding the device in one location while standing outdoors can result in the latitude/longitude wandering over a reasonable distance. Even if you set the phone on a rock the results can wander. Just keep this in mind if you have accuracy requirements. Just because the user is standing still outdoorsÂ doesn’tÂ mean that the indicated location will stay exactly the same. It could wander over a 10 to 50 meter radius or more.

I have occasionally seen wild location fluctuations that were enough to make a mapping application nauseatingly jump back and forth between different center points. Unfortunately, when it happened IÂ didn’tÂ have the device on a debugger, but I suspect it had to do with the phone detecting various WiFi end points when I stopped at stoplights along my route. Itâ€™s possible the phone tried to resolve those WiFi locations using the Google Network Location service and that there was some lag time in processing those results.

Are Android, off-the-shelf, retail smartphones as accurate as something like a Trimble Pro Series Receiver? No way, not all, definitely not! If you have high accuracy, field-usage requirements calling for sub-meter results you should not be using a typical retail Android device. For example, if you are standing in an intersection with four manhole covers that are two feet apart and form a circle, you could not accurately map which manhole cover is the right one that you need to be working in if you are using an off-the-shelf Android. Using my own phone as an example, on a good day my Samsung Galaxy SIII occasionally has down to 3 meters GPS accuracy for short periods of time and then it can start wandering.

Comparing Network and GPS locations and using Criteria. Yes, you shouldÂ definitelyÂ compare the results between these two. The more data you have the better. There’s not a whole lot to say about this other than look at timestamps, accuracy and distance traveled factors. In my experience, just something to keep in mind is the network locations happen much slower than the GPS. I expected this because of the lag time involved with the phone sending information to the remote location service and then waiting for a result to come back. You can read more about how location services work in this article.

The use of android.location.Criteria can also be used to control which listener(s) are used. You can experiment with Criteria using the GPSTester tool. The only minor caveat is it doesn’t include all possible criteria in the current version (v1.2.1.1). My general recommendation is to skip Criteria and hard code in the validation rules. I don’t currently subscribe to the idea that the Criteria will know what’s best for my end users. Please comment below if you feel otherwise, but in my own experience I haven’t come across a use case where Criteria gave what I thought was the right answer. In one use case, the Criteria was set to not incur costs, yet the value returned was to use the network provider even though GPS was available. Using the network location service would have incurred bandwidth costs.

What to do when app is minimized? A common workflow is for a user to minimize the app and then either forget about it or come back to it later. I’ve included this because it’s common for the user to get a phone call or a text message that results in the app being minimized. In most of the use cases I work with, the requirements call for both the GPS and Network Location services to be shut off when the app is minimized. Itâ€™s just way too easy for the user to forget about the app and then it could kill the battery in a very short time.

There is the option of using a passive listener. I rarely use these because it assumes some other unknown application will spin up the GPS or Network Location providers. Â Your use cases will help you decide whether or not implementing a passive listener is a good idea. If your target audience is a 20-something student who is constantly using location based social media such as Facebook every hour and eighteen hours a day then thereâ€™s a good chance the passive listener will return recent and mostly accurate results. However, if you are building a commercial-grade application on a work-related phone then a passive listener is significantly less likely, or even highly unlikely, to speed up the applications ability to get a fast fix on startup.

Use Intents when implementing a passive listener, this involves adding directives in the AndroidManifest.xml as well as writing some code. A great example of how to do this can be found here, so Iâ€™m not going to reinvent the wheel. Just donâ€™t forget to add the receiver tag to your manifest:

<receiver android:name=".receivers.PassiveLocationChangeReceiver"/>

When the app restarts you can use information from the passive listener and consider whether or not it meets your accuracy criteria. You can also use the passive listener to run background processes within your app while it is minimized. You’ll use the Location timestamp once the app is opened again. If the timestamp is very recent, such as less than one hour old, then you could consider using the passive location immediately. Otherwise, you’ll be back to waiting for a real-time location result. For this approach consider comparing Location.getRealtimeNanos() to SystemClock.elapsedRealtimeNanos(). If your target audience are business travelers, then for example you might want to reject cached results that are greater than 8 hours old because chances are the user has hopped on a plane.

Privacy, data storage and data consumption. Yeh, itâ€™s a bummer I even had to bring this up. Iâ€™m not a lawyer, but every lawyerÂ I’veÂ worked with on these types of production applications strongly reminds me to have a Privacy Policy and to be transparent about how the data is stored and how the data is used. Donâ€™t try to do this on your own, hire a lawyer or the equivalent of a lawyer in your country, as the case law is constantly changing and can vary from State-to-State and Country-to-Country.

And, that’s the wrap-up for Part 2. Stay tuned for additional posts on this series that cover using the GPSTester tool and how to interpret the results.

There are many different ways for developers to get access to location information using the Android SDK, and there are already plenty of code examples on the internet showing how to do that. This post, on the other hand, focuses on putting the location data into context and offers suggestions on how to use it in the best possible way. In this series Iâ€™ll discuss various aspects of the Android SDKâ€™s android.location package and how you can best take advantage of them to build a great end-user experience.

To try out GPS different usage scenarios, be sure to download the GPSTester app, both the .apk and source code are available on github.

Six types of location data

Itâ€™s not just about latitude and longitude. The great thing about the Android SDK is you get access to a ton of information related to location. The fun, challenging and sometimes frustrating part is deciphering the information. There are six types of location data that you have programmatic access to. This data comes from whatâ€™s referred to as a provider, and it will be either a cellular network-based or a GPS provider. As youâ€™ll see, you can use these providers in a variety of different ways.

All locations derived from the LocationManager are guaranteed to provide a valid latitude, longitude and timestamp, all the other parameters are optional depending on who manufactured the handset and who is providing cellular service. However, if you have a device that doesnâ€™t give you access to the accuracy parameter then the location data is practically worthless. I havenâ€™t come across such a device yet, but Iâ€™m assuming itâ€™s possible. My advice: always check for null location parameter values in your production code.

Cached GPS. Most Androids have the ability to store the last know GPS location. This is typically used when an application first starts up, and you can retrieve the timestamp, latitude, longitude and accuracy.

locationManager.getLastKnownLocation(LocationManager.GPS_PROVIDER);

Cached Network. Android devices can also store the last known location as determined by the cellular carrierâ€™s network location provider. The provider gathers information from the cell network and WiFi, if itâ€™s turned on, then sends that off to a remote server-side process that crunches the information and sends back an approximate location. This is not available in all countries. Â Just like the GPS, youâ€™ll typically retrieve the timestamp, latitude, longitude and accuracy.

Real-time GPS. This is the raw information that is streamed off the GPS.Â When a GPS is first turned on it wonâ€™t immediately return any information, it has to basically warm up first. The warm up time varies by device and can typically take from one minute or longer if you are inside a building. More on that in a bit. Depending on what your provider allows you can get access to timestamp, latitude, longitude, altitude, bearing, speed, accuracy and distance travelled.

Real-time Network. This is the raw network location provider information returned by the cellular carrier, such as AT&T in the U.S. Different carriers use different information to determine location such as WiFi data, GPS information, nearby cell towers, etc. Depending on what your carrier allows, you can get access to timestamp, latitude, longitude, altitude, accuracy and distance travelled.

Passive. This option just means that your application can listen for location updates while it is in a minimized state. The idea is to save battery power, such that your applications providers arenâ€™t running full speed ahead while the app is miminized. That would be huge battery drain. Passive location can listen for another application to request location updates. You may be able to get access to timestamp, latitude, longitude, altitude and accuracy. As far as I know, you wonâ€™t be able to determine from which provider this information was derived.

NMEA. Although itâ€™s not human readable, you can get access to the raw NMEA strings. Typically these strings are used for programmatic access and would only make sense to a developer or engineer. This data is often used in maritime apps. The data is only available once the GPS has warmed up, which is a concept discussed in more detail below.

addNmeaListener(GpsStatus.NmeaListener);

Working with cached locations

Working with cached locations is an interesting problem because the first location you get is not going to be the most accurate, and typically it can be wildly inaccurate compared to your actual, physical location. Think of the location capabilities on the device as being similar to a car engine on a cold morning. The engine needs to warm up first before it can perform optimally, and this typically takes a few minutes. And once the engine is warmed up, if you make a quick stop at the grocery store the engine will still be mostly warm when you come back out. Â

Location provider components act in a very similar way. When an app is first launched the providers may be â€ścoldâ€ť. The only information that is immediate accessible will be via cached network, cached GPS and the passive listener. Â Alternatively, if the app is launched when the providers are still warm or even hot, then you will get better accuracy with the cached results and the real-time locations may start streaming sooner.

Cached locations arenâ€™t always best 1st choice. When an app first launches, the inclination of many developers is to simply grab the cached location data and use that as a starting point, but as implied aboveâ€¦be careful! Caveat emptor applies here. Cached locations are simply the very last result stored on the phone and itâ€™s not always what you expect. As you can see from the Cached Network Provider screenshot taken from the GPSTester app, the accuracy of that data snapshot is 1780 meters (1.1 miles). Keeping that in mind, consider these use cases:

Business traveler – You last opened the application on a business trip to San Francisco and you are now in Denver where you happen to live. When you start the app in Denver the cached locations could very well be for San Francisco.

Consumer â€“You last used the app at home and now you have traveled to the other side town, or even to a nearby city to visit friends. The last cached locations will be from near your home and not very helpful to your immediate surroundings.

Cached locations vary considerably because you almost always never leave the location provider services running constantly. Leaving them on all time can easily kill the battery in a few hours, and end users will unanimously reject that kind of battery life.Â Therefore cached locations represent the fact that either your app or another app had explicitly turned on location providers at some point in the past.

Timestamp and accuracy. You will also want to compare the timestamps and accuracy of the cached network and cached GPS. If network data isnâ€™t available then look at the cached GPS results. If both are available then check to see which one is the most recent. Â Here are several example use cases to demonstrate whether or not they would be valid for your requirements. Which one would you go with?

Date of cached network data is 7 days old and accuracy is greater than 1000 meters (or ~3/4 mile). No cached GPS data is available.

Timestamp of cached GPS Â is the yesterday with accuracy of 1000 meters, and cached network data is from today with an accuracy of 1780 meters.

Timestamp of cached GPS is from yesterday and the accuracy was 250 meters. Cached network data is not available.

When you answered those scenarios did you take into account any behavior patterns? How would your answers differ if you were aiming for business travelers versus aiming for local consumers that frequently stay within a 50 mile radius?

Know Your Users Behavior Patterns.Â The bottom line is people move around a lot. Youâ€™ll need to be very aware of theÂ personaÂ that you are targeting with your app and understand their general behavior and geographic movement patterns that answer questions such as:

How often do they access the map from a different location: once a day, several times a day, once a month?

Are their trips always within a dense city limits? City center users may encounter many more stop lights and traffic jams in town but drive shorter distances. Urban users who are out in the suburbs might travel longer distances.

Do they travel or live near many canyons, mountains, tunnels or large buildings that might affect an accurate signal?

What country do they live in? Not all countries offer network provider services.

Are the use cases so varied that you simply can’t predict the behavior patterns? If this is the case then you may have to launch the app and build in ways to gather statistics so you can start to analyse the behavior.

As you can see, someone’s geographic behavior of a period of time canÂ significantly affect the model you are using and force you to look at cached location data in new and different ways.

Wrap Up

Thatâ€™s it for Part 1 of this series. We’ve looked at six different ways that Android can provide location information, and examined different scenarios on how you can use that information. We also briefly skimmed the surface on how geographic behaviors can affect what you do with cached location results. You’ve seen that cached results can be misleading and your users may have to wait until the phone can provide more accurate information. I hope you find this information useful. Stay tuned for Part 2 which will talk about different patterns for retrieving real-time data, and how to start applying use cases to validate your results.

TheÂ Android GPS Testing Tool is a must have tool for any Android developer using GPS or network location data. It’s intended to help developers, hobbyists and scientists understand the internal workings of smartphones and tablets so that they can build better applications.

Itâ€™s completely configurable via preferences so that you can quickly and easily cook up scenarios and test them out. Use it to compare cached results versus real-time data as well as plot all of it on a map. Â The source code is included to see you can see how all of it works, and if you want to get started immediately simply grab the .apk file. So have at it and let me know what you think and what could be improved.

What are some example scenarios that I can test with this app?

Test how long it takes for GPS to provide it’s first update after the app is started

Compare and contrast Network locations with GPS locations

Study cached network and GPS data that is provided at app startup.

What data does it provide? Hereâ€™s an overview of the raw data you’ll be able to see: Â

What are some of the configuration options? Here are some of the configuration options:

Use GPS and/or Network provider data

GPS minimum update time

GPSÂ minimum update distance

Network minimum update time

Network minimum update distance

Use Criteria such as accuracy, power and cost

I built v1.0 out of necessity to gain an understanding of the complexities and subtleties of the android.location package across different devices. No blog post can comprehensively explain how each individual smartphone or tablet will work as related to Location, GPS, GPSStatus, GPSSatellite, Criteria and LocationManager using different providers such as GPS and Network.

Furthermore, when I was first starting out building Android apps, I found all the different options confusing in terms of what they did and how to easily compare results both literally and on a map.

My wish list for future updates includes:

Fully flesh out the ability to use all of the Criteria settings. V1.0 uses a subset of all the possible Criteria.

While the code for shutting off the LocationManager on Android is straight forward, sometimes just one misstep and you can spend hours trying to figure out why it doesnâ€™t work. What will happen is you will see in your code that the LocationManager and perhaps even the listeners will be null and the GPS active icon will continue to display and your battery will drain faster than you ever thought possible. There are a bunch of comments already on Stack Overflow about this problem, so this post attempts to consolidate answers and provide some straight forward steps to help with the most common mistakes.

First letâ€™s review the steps for properly shutting LocationManager down:

Step 1. Remove the listener or listeners. If you are using both GPS and network providers you have to remove both. It sounds oversimplified, but you can only remove the exact listener that is associated with a particular instance of the LocationManager. Itâ€™s a one-to-one relationship.

Step 2. Set LocationManager to null.

Step 3. Set listeners to null.

Hereâ€™s what the code looks like, the names should be self explanatory:

If these steps donâ€™t work and the GPS icon continues to display, then there are potentially two reasons why:

Reason 1. You have instantiated intentionally or accidentally more than one copy of LocationManager and/or a listener. This is by far the most common and frustrating reason. The easiest way to avoid this mistake is to only start the LocationManager via the onResume() event, and shut it down using onPause(). Carefully control what is allowed to start and stop it. Be mindful when passing around instances of LocationManager between different Activities whether it is a singleton, static class or simply a reference to the actual object. If LocationManager is already running and you spin up a new instance, then you may not be able to shut it off programmatically and youâ€™ll have to manually shut it off.

Reason 2. Something else other than your app is causing the LocationManager to run. This can easily give the appearance that it has not been turned off via your app. Repeated testing can eliminate this as a possibility, and to make sure you can shut off all other non-essential applications that use location.

Under low accuracy situations, Iâ€™ve noticed on both iPhone and Android that the GPS location can fluctuate wildly over a short period of time, sometime jumping 2500 ft (762m) or more within several seconds. By low accuracy I mean the GPS result indicated a horizontal accuracy of greater than 1000ft (305m). This creates a really poor user experience, so I quickly implemented a very rough algorithm to minimize these fluctuations. Here’s some psuedo-code that demonstrates the concept:

The theory is that the distance and time between fluctuations calculate out to speeds greater than 2500 ft/1 second. You can adjust the parameters as you see fit. I also assumed that under normal driving (or walking!) conditions you wouldnâ€™t be going that fast, right?

You could also write a native extension, but I didnâ€™t have enough time to do that. The native Android SDK offers many ways to use the GPS that could eliminate these problems.

This could certainly use some tweaking, so if you have suggestions for improvement please post a comment or send me an email!