Thinking about Science, Engineering, and Society

March 26, 2010

Meet the Users

Our application has been on the market now for a couple days. And so now, we have something new: users.

One thing I learned long ago about users: They don't always behave the way you expect or want!

One of the little not-so-endearing features of Google's Market application store, is that the descriptions you get are pathetic. You get to use 325 characters and two screenshots. That's it. Somehow, in 325 characters, you have to tell users what your application does, and why they'd like to try it.

This, and the ONE comment you get to submit as a purchaser of your own app, are your only means of communicating with your customers.

So ours reads:

SmartVolume by SF Smart Systems makes it easy to get the most out of your phone.

One screen to manage all your volume settings and even lock volume for each type of sound. No more fumbling around for a setting while trying to hear a caller.

Never miss an important call because you accidentally turned down the ringer.

As a major purchaser of phone apps, I've read a lot of user comments on apps, both on the iPhone (all four of us in my family have one), and on the Google Market application.

As a purchaser, I've found these comments to be all over the map. A few are helpful. Many are just insults, obscene comments, and other juvenile conduct. Others are obviously confused about either their phones or the app's intended purpose.

And many clearly show that the user has not read the app description before downloading, trying, and commenting. Clearly-labeled prank apps being taken for the real thing. People expecting the obviously impossible.

A report that likes a feature, but complains it won't let her change something on her phone. I'm not sure exactly what she means, but I'd like to know! She also reports the FAQ link doesn't work -- but nobody has asked any questions yet.

A guy who really loves it. (Thanks!)

A "Nice try", but reports that while it acts like it'll work, it doesn't -- whatever that means. He reports he's on a Motorola Droid

A guy who reports it works great on his Motorola Droid

OK, at this point, I'm scratching my head. Do I have a bug? No sign of any problems examining the reports from Flurry. There are a few exceptions that I handle and report, because it came up exactly once during my stress testing. Thanks to Flurry, I can see that it happens once every several thousand passes through that code -- about once per 1000 sessions.

But then I get one more comment.

A guy who reports it works, but then forgets, but works again if you fiddle with the volume control

OK...this may be a clue. I go back and retest.

After some testing, uninstall, reinstall, test, retest, reboot, I come to understand that the problem only exists between install and first boot. Once you reboot your phone once, it works reliably and properly.

But if you get a low memory situation, and the system kills your app, THEN it does not restart.

When I tested this behavior -- first I tested that it starts up on booting OK. Then I tested that if you kill it, it restarts properly. That worked. But I needed to have tested them in the other order.

With this information, it didn't take long to track down and fix, and I had a new version up on the Market an hour or so later.

(The problem was that the user interface only bound to the service, which causes it to run, but didn't actually "start" the service, which is required to keep it running even after a low memory situation).

But what to do about those comments on the old version? I'd love to have the people retry, and update their comments, but I have no way to communicate with them.

So I just added the phrase to my comment, "NEWLY UPDATED!". And I found this nicely moved the comment up to be more recent than the other comments. Thus, this now marks where the comments are obsolete!

I tried really hard to test low memory. I really did.

I blew it. In my rush to get a fix up there, I neglected a necessary check. Fortunately, a few hours later I happend to be trying UStream Viewer, and it caused a low memory situation. My app blew a fuse (FC - Force Close). A quick look revealed the problem, and up went a second revision.

A quick check of the bad version (1.0.293) showed:

Uh, oh. Well, nothing I can do but hope they upgrade. So this time, I added the version number to my comment as well. "NEWLY UPDATED! 1.0.296"

This is a small percentage of the user base, but I don't want any unhappy customers, even for a free app. The good news is, very few people are encountering low memory situations. As part of my update, I made Flurry report low memory situations. Out of 1206 sessions over the past couple days, only 6 have encountered a low memory situation at all.

I'm going to keep doing this trick with updating my market comment, both in a futile attempt to communicate to users who've already posted comments, and to let potential users know which comments are still relevant.

But the other thing I've learned, is I Really Need A Tool to really test low memory. If I can't find one, I'll have to write it.

So that's all the market comments we've received. Our rating is at 4 out of 5 stars, which isn't too bad for a first attempt, and a few glitches.

But what about email? Did we get any email? The Market app makes it easy to send email -- just tap the Email Developer link.

Yep. We got email. After the above Market comments, two emails came in.

The first was from a fellow Android developer, commiserating about a post about getting useful user feedback. She liked the app.

The second was from an old friend, whom I'd worked with at MIT and Symbolics back in the 1970's/1980's. Howard I. Cannon is cofounder and CTO of ModelSheet Software which is in public beta now. It generates nice, easily customizable Excel spreadsheet templates for various business modelling needs. A very clever idea. Since business is addicted to Excel, but it becomes unweidy -- do the customization separately, rather than by editing the spreadsheets directly.

Howard had some great feedback for me. I was supposed to call him back today for more, but life intervened, but we'll talk more this weekend. And I was able to return the favor by looking over his beta site as well.

But that's it. The only email has been from other developers!

That's really not that surpising. We developers understand how valuable feedback is. Even if it annoys, it's our lifeblood. Customers, especially for free or inexpensive apps, often don't have so much investment in the product. To them, a problem is just an annoyance.

But it's really a partnership. If you take a moment to contact a developer with a problem, you'll help make a better product!

I've done everything I could think of, to make it easier for users. I've used Flurry to automate reporting of errors and other things that will help me make the application better. I've put a menu item in the app to take them to our support page, where they can report problems. I've put in an extensive tutorial (TOO extensive, reports Howard, and I agree) that covers every aspect of the program, and the ins and outs of volume controls on the Android platform.

But it'll never be enough.

Remember all this, the next time you're reading user comments in a phone app store.

I really hope Google folk read this. The Market really needs some work, and #1 on my list is to improve the communications between developers and users. By contrast, the iPhone app store allows over twice the communications and a lot more tools.

Comments

You can follow this conversation by subscribing to the comment feed for this post.

Hi Bob - nice, interesting post.

As a fellow developer I saw your post (mentioning SmartVolume) on Android Developers and, since IMHO you provide excellent posts on that forum, I checked out your app.

First of all, well done for a great app!

I don't envy you, though! You're taking on an area which is rather complex (too bothersome) for most people to understand. Going through the tutorial I felt a little overwhelmed. I don't like reading much on small screens, but I kind of felt I had to to appreciate the app.

In the end, I skim-read the tutorial and so only have a vague understanding of how it works. I never realised about linking audio volumes before.

The app does look very useful, but I decided to uninstall it for two reasons:

1. I mainly really use my phone for development and so don't have much of a need for it.

2. I'm a little nervous that apps using services will (may) kill my battery life.

Also, I'm a little put off by an extra notification icon. Yes, I know you can turn it off. I just mean that generally I don't like extra notification icons though that importance is tied to my point (1). I realise there is not much you can do about that!

Great idea to put a comment in each time you do an update. FWIW, you can see all comments (not just your latest one) in cyrket. Of course, 99% of users will never go to that website but it can be useful for yourself to see where your updates fit into the comment list. Also, there you'll see comments in all languages.

One of my apps has over 30,000 downloads and I've received less than 10 emails with bugs/suggestions. I now include an About page which has links back to the Market "details" page (so they can easily leave feedback or uninstall) as well as replication of the Market's "View more applications", "View application's web page", "Send email to developer".

My app averages 4.50, but yesterday I got my first 1-star from someone complaining that it requires an internet permission "only to display ads". Hmmmmm...

When I get a bad comment (and I feel its harsh), I reply with my own comment. Always polite of course :) It helps to push down the bad comment too.

Mark, one thing I forgot to mention in my reconstructed post (it was in the original) was that Howard mentioned the same issue with the tutorial. I too was concerned, but the initial feedback from my partner-in-crime was that we should go with it.

The basic issue is figuring out information users are interested in, and you need users for that -- and you need the users to be aware of the information before they can tell you!

So if you have any feedback on that, it'd be great! Otherwise, I'll try to deduce it from the Flurry data and seat-of-the-pants intuition.

I wonder if it would be worth asking the user in the tutorial if they'd like the notification icon? Here's where the difference between a developer's perspective and a user's perspective may differ. Or may not.

Thanks for the cryket suggestion. I hadn't even realized that non-English comments don't appear on the Market. I've read Japanese comments in the iPhone App Store -- though that may be because I have the Japanese IME set up on my iPhones.

Great suggestion for the market link -- and I should have thought of the "email developer" link, rather than sending them to the web site. Especially since our web site development is lagging behind at the moment.

Finally -- about services and battery life -- properly coded, a service won't kill the battery. The service only responds to the specific broadcast events -- in this case, the volume control changes. It doesn't consume any power otherwise.

This is something I've kept a close eye on.

The downside it DOES have, is that it consumes a bit of memory -- and if the system kills it to make room for other apps, during that interval it won't be able to respond to changes. This is one of the tradeoffs we have to make on a limited device.

Still -- the device isn't so limited. I have running in a simulator on my Nexus One the very operating system (ITS) that Howard and I used at the MIT AI/LCS labs back in the 1970's. And it runs a lot faster, too!

But it's a good sign that I'm having trouble triggering a low-memory situation even with all the stuff I've got on my phone! And that so few users are encountering it. (And that they generally don't even know they're encountering, except for ones who also happen to be in the unfortunate 67).

Again, thanks for your comments. No hard feelings about the uninstall -- I don't imagine it would be all that useful on a purely development phone.

Quick question: I'm inferring that there are no low-memory tools available in the current Android development environment. Is that the case?

I only ask because I do iPhone App dev (personal use only -- yes, I pay $99/year to write tools for myself... sigh) and the iPhone simulator has a low-memory emulation (to an extent). I'm just wondering if Android has *nothing* or if it's more along the lines of the iPhone simulator.

@Daniel -- I should find out more about the iPhone's low memory tools and handling to properly compare the situations.

Since the iPhone doesn't support multiple simultaneously active applications (as of 3.x) the comparison isn't obvious to me.

I suspect the idea of a "low memory tool" would be analogous only at a very abstract level -- and that the answer to your question is "nothing". Whether it's more or less important than on the iPhone, I couldn't say, but I think there should be one.