31 January 2014

Android 4.4 KitKat introduced a new system service called procstats that helps you better understand how your app is using the RAM resources on a device. Procstats makes it possible to see how your app is behaving over time — including how long it runs in the background and how much memory it uses during that time. It helps you quickly find inefficiencies and misbehaviors in your app that can affect how it performs, especially when running on low-RAM devices.

You can access procstats data using an adb shell command, but for convenience there is also a new Process Stats developer tool that provides a graphical front-end to that same data. You can find Process Stats in Settings > Developer options > Process Stats.

In this post we’ll first take a look at the Process Stats graphical tool, then dig into the details of the memory data behind it, how it's collected, and why it's so useful to you as you analyze your app.

Process Stats overview of memory used by background processes over time.

Looking at systemwide memory use and background processes

When you open Process Stats, you see a summary of systemwide memory conditions and details on how processes are using memory over time. The image at right gives you an example of what you might see on a typical device.

At the top of the screen we can see that:

We are looking at that data collected over the last ~3.5 hours.

Currently the device’s RAM is in good shape ("Device memory is currently normal").

During that entire time the memory state has been good — this is shown by the green bar. If device memory was getting low, you would see yellow and red regions on the left of the bar representing the amount of total time with low memory.

Below the green bar, we can see an overview of the processes running in the background and the memory load they've put on the system:

The percentage numbers on the right indicate the amount of time each process has spent running during the total duration.

The blue bars indicate the relative computed memory load of each process. (The memory load is runtime*avg_pss, which we will go into more detail on later.)

Some apps may be listed multiple times, since what is being shown is processes (for example, Google Play services runs in two processes). The memory load of these apps is the sum of the load of their individual processes.

There are a few processes at the top that have all been running for 100% of the time, but with different weights because of their relative memory use.

Analyzing memory for specific processes

The example shows some interesting data: we have a Clock app with a higher memory weight than Google Keyboard, even though it ran for less than half the time. We can dig into the details of these processes just by tapping on them:

The reason that Clock has been running at all is because it is being used as the current screen saver when the device is idle.

Even though the Clock process ran for less than half the time of the Keyboard, its ram use was significantly larger (almost 3x), which is why its overall weight is larger.

Essentially, procstats provides a “memory use” gauge that's much like the storage use or data use gauges, showing how much RAM the apps running in the background are using. Unlike with storage or data, though, memory use is much harder to quantify and measure, and procstats uses some tricks to do so. To illustrate the complexity of measuring memory use, consider a related topic: task managers.

Understanding task managers and their memory info

We’ve had a long history of task managers on Android. Android has always deeply supported multitasking, which means the geeky of us will tend to want to have some kind of UI for seeing and controlling this multitasking like the traditional UI we are used to from the desktop. However, multitasking on Android is actually quite a bit more complicated and fundamentally different than on a traditional desktop operating system, as I previously covered in Multitasking the Android Way. This deeply impacts how we can show it to the user.

Multitasking and continuous process management

To get a feel for just how different process management is on Android, you can take a look at the output of an important system service, the activity manager, with adb shell dumpsys activity. The example below shows a snapshot of current application processes on Android 4.4, listing them from most important to least:

There are a few major groups of processes here — persistent system processes, the foreground processes, background processes, and finally cached processes — and the category of a process is extremely important for understanding its impact on the system.

At the same time, processes on this list change all of the time. For example, in the snapshot above we can see that “com.google.android.gm” is currently an important process, but that is because it is doing a background sync, something the user would not generally be aware of or want to manage.

Snapshotting per-process RAM use

The traditional use of a task manager is closely tied to RAM use, and Android provides a tool called meminfo for looking at a snapshot of current per-process RAM use. You can access it with the command adb shell dumpsys meminfo. Here's an example of the output.

Example output of dumpsys meminfo command, showing memory currently used by running processes.

We are now looking at the same processes as above, again organized by importance, but now with on their impact on RAM use.

Usually when we measure RAM use in Android, we do this with Linux’s PSS (Proportional Set Size) metric. This is the amount of RAM actually mapped into the process, but weighted by the amount it is shared across processes. So if there is a 4K page of RAM mapped in to two processes, its PSS amount for each process would be 2K.

The nice thing about using PSS is that you can add up this value across all processes to determine the actual total RAM use. This characteristic is used at the end of the meminfo report to compute how much RAM is in use (which comes in part from all non-cached processes), versus how much is "free" (which includes cached processes).

Task-manager style memory info, showing a snapshot of memory used by running apps.

Task manager UI based on PSS snapshot

Given the information we have so far, we can imagine various ways to present this in a somewhat traditional task manager UI. In fact, the UI you see in Settings > Apps > Running is derived from this information. It shows all processes running services (“svc” adjustment in the LRU list) and on behalf of the system (the processes with a “<=Proc{489:system/1000}” dependency), computing the PSS RAM for each of these and any other processes they have dependencies on.

The problem with visualizing memory use in this way is that it gives you the instantaneous state of the apps, without context over time. On Android, users don’t directly control the creation and removal of application processes — they may be kept for future use, removed when the system decides, or run in the background without the user explicitly launching them. So looking only at the instantaneous state of memory use only, you would be missing important information about what is actually going on over time.

For example, in our first look at the process state we see the com.google.android.apps.magazines process running for a sync, but when we collected the RAM use right after that it was no longer running in the background but just being kept around as an old cached process.

To address this problem, the new procstats tool continually monitors the state of all application processes over time, aggregating that information and collecting PSS samples from those processes while doing so. You can view the raw data being collected by procstats with the command adb shell dumpsys procstats.

Seeing memory use over time with procstats

Let’s now go back to procstats and take a look at the context it provides by showing memory use over time. We can use the command adb shell dumpsys procstats --hours 3 to output memory information collected over the last 3 hours. This is the same data as represented graphically in the first Process Stats example.

The output shows all of the processes that have run in the last 3 hours, sorted with the ones running the most first. (Processes in a cached state don’t count for the total time in this sort.) Like the initial graphical representation, we now clearly see a big group of processes that run all of the time, and then some that run occasionally — this includes the Magazines process, which we can now see ran for 3.6% of the time over the last 3 hours.

The percentages tell you how much of the overall time each process has spent in various key states. The memory numbers tell you about memory samples in those states, as minPss-avgPss-maxPss / minUss-avgUss-maxUss. The procstats tool also has a number of command line options to control its output — use adb shell dumpsys procstats -h to see a list of the available options.

Comparing this raw data from procstats with the visualization of its data we previously saw, we can see that it is showing only process run data from a subset of states: Imp Fg, Imp Bg, Service, Service Rs, and Receiver. These are the situations where the process is actively running in the background, for as long as it needs to complete the work it is doing. In terms of device memory use, these are the process states that tend to cause the most trouble: apps running in the background taking RAM from other things.

Getting started with procstats

We have already found the new procstats tool to be invaluable in better understanding the overall memory behavior of Android systems, and it has been a key part of the Project Svelte effort in Android 4.4.

As you develop your own applications, be sure to use procstats and the other tools mentioned here to help understand how your own app is behaving, especially how much it runs in the background and how much RAM it uses during that time.

09 January 2014

The latest release of Google Play services is now available on Android devices worldwide. It includes new Turn Based Multiplayer support for games, and a preliminary API for integrating Google Drive into your apps. This update also improves battery life for all users with Google Location Reporting enabled.

You can get started developing today by downloading the Google Play services SDK from the SDK Manager.

Turn Based Multiplayer

Play Games now supports turn-based multiplayer! Developers can build asynchronous games to play with friends and auto-matched players, supporting 2-8 players per game. When players take turns, their turn data is uploaded to Play Services and shared with other players automatically.

We are also providing an optional new “Connecting to Play Games” transition animation during sign-in, before the permission dialog appears. This helps contextualize the permission dialog, especially in games that ask for sign in on game start.

Google Drive

This version of Google Play Services includes a developer preview of the new Google Drive API for Android. You can use it to easily read and write files in Google Drive so they're available across devices and on the web. Users can work with files offline too — changes are synced with Google Drive automatically when they reconnect.

The API also includes common UI components including a file picker and save dialog.

Google Mobile Ads

With Google Play services 4.1, the Google Mobile Ads SDK now fully supports DoubleClick for Publishers, DoubleClick Ad Exchange, and Search Ads for Mobile Apps. You can also use a new publisher-provided location API to provide Google with the location when requesting ads. Location-based ads can improve your app monetization.

Google+

An improved Google+ sharing experience makes it even easier for users to share with the right people from your app. It includes better auto-complete and suggested recipients from Gmail contacts, device contacts and people on Google+.

More About Google Play Services

To learn more about Google Play services and the APIs available to you through it, visit the Google Services area of the Android Developers site. Details on the APIs are avaialble in the API reference.