I'm planning to release a project which will integrate with several popular content management systems as a plugin and operate as a stand-alone service - beyond providing users with a simple contact form which posts to a ticketing system, what are the most effective ways you have found to solicit useful feedback from within the interface and manage user feedback?

Edit:

@Nir & @Rahul - The scope of user feedback is obviously not going to be limited to UI suggestions (or bug reports, or ...) but I'd like to know whether there are any good ways to ferret out commonly-held problems with the application's UI, localization, et cetera. (Note that commonly-held problems might only be common for a limited subset of users - and, unfortunately, I don't foresee having the option to sit down and test every UI localization with a native speaker, for example)

I'm planning a "bug report/suggestion" modal which is accessible from any page and reports the page it was called from and I want to get as much information on the problem as possible. I'd expect that most users would need a nudge in the right direction to get usable feedback without any back-and-forth.

It's a two-part question, maybe it reads better as:

How do I encourage remote users of a web application to offer suggestions above and beyond "It doesn't work" directly through the application, rather than in discussion on a forum, plugin comments page, et cetera?

What is the best way to manage feedback aggregated from multiple sources? (when that inevitably occurs despite all best efforts to prevent it)

Edit w/Answer Accepted:

Thank you, Rahul, Nir, and indolering - I think I have a few of the missing pieces to the puzzle of comprehensive testing.

Given the recommendations I've received and my previous plans, my present strategy looks something like this:

Solicit feedback from within the UI
with a simple contact/suggestion/bug
report form which posts to a
ticketing system

Provide a forum for users to post
suggestions and get support from
eachother (I'll probably start out
spending a lot of time there myself)

Post a working demo version of the
application, integrate
click-tracking tools to watch how
users interact with the interface

As the project matures, offer users
incentives of some form to
participate in beta testing and
offer feedback

Consider third-party feedback
collection and organization tools
like UserVoice, GetSatisfaction, et
cetera as appropriate to demand (I'm
thinking that the choice of solution
will depend a lot upon the type of
userbase the project draws - given
that I'm planning to see shared
hosting users install my
application, it might make sense to
use a third-party service versus
rolling my own... but I'll wait to
solve that problem until I get to
it)

4 Answers
4

Feedback

We use UserVoice for feedback. Users get 10 votes each and can create or vote on something (an idea, bug report, critique, etc) and assign up to 3 votes to each item. Once they're out of 10 votes they can't do anything until we close or complete an item they voted on, at which point their points for that item are refunded. It works great as a way to collect general ideas and feedback from people while allowing users to vote on each other's ideas. The UI could use some work but it's a nice service, especially since the free version is more than enough.

Customer support

GetSatisfaction works great as a customer support forum where you can get feedback but also allow users to help each other. It's great when you want to take a step beyond UserVoice and let the community evolve a little bit. People can offer feedback categorised by bugs, ideas and compliments. They can also vote up each other's items and indicate how they feel about it using a smiley face. It's a nice way to get an immediate impression per item of how people feel: if there are 22 sad faces, you're probably more motivated to take on this problem than a bunch of indifferent ones. The UI is pretty nice, although a little noisy and distracting when compared to something like UserVoice.

Ticketing email integration

Beyond that, consider allowing people to email you. Most ticketing systems will allow you to set up forwarding email addresses. The nice thing about email integration with ticketing is that you can keep the entire conversation with the user in email by replying back and forth, which brings the convenience and familiarity with UI to the user instead of expecting them to learn a potentially confusing ticketing system UI. Fogbugz does a good job of this and I've heard great things about Lighthouse.

Why not just use a forum or contact form?

Today's Web 2.0 era has a lot of solutions that are specific to certain situations. It makes sense to try and take advantage of a user experience designed exclusively for tackling the problem of "I built something and I want to collect feedback". You can always build your own solution, but you probably won't be able to spend enough time on it for it to be as good as products like the ones I mentioned above. Companies like 37signals, who consider customer support a major priority, have built their own superior solutions (for instance, Basecamp Answers).

Open source alternatives

My experience is with the above solutions, but I've seen mention of some open source alternatives for gathering feedback:

IdeaTorrent - this looks like an interesting cross between a forum and something like UserVoice. Development of the new version has stopped, however

For getting UI feedback do "hallway usability testing" - just grab a random victim, hm, I mean person that was unlucky enough to pass nearby (friend, family, coworker, delivery man whatever, just not another developer working on the same system) have him/her sit in front of your computer and complete some random task with the app while you are watching.

You can also use screen sharing software (Skype can do that, or you can use something like TeamViewer or GoToMeeting) to watch real users remotely when they are using the system or use a service like UserTesting.com

You need a surprisingly low number of such tests to find the problem that really matter (that are preventing users from completing the task or annoy users).

Also, A/B testing is a really good tool to use, always have tests running, if you don't know what to test just swap out random pieces of the UI and replace them with something completely different - if the original UI doesn't outperforms the replacement it's not optimal (or it just doesn't matter, not every piece of UI is critical after all).

From my experience selling commercial software (so my customers probably expect more support than yours) asking for feedback just doesn't work, at least for usability, once you fix the obvious problems that you already know about - especially if you use the system yourself - all UI-related feedback just stops.

For managing the problems you find any ticketing system will do - but it's important to prioritize the problems by severity and solve problems in order of priority, otherwise you are likely to implement the easy fun solutions to the small problems first.

Hmm, interesting that you interpreted his question as one about user testing, I hadn't read it that way
–
Rahul♦Oct 19 '10 at 10:34

1

@Rahul - I'm not sure it's a question about user testing (I know you've seen my comment on the question so you know I'm not sure what the question is about) - if the question is "how to get and manage feedback about UI" than the answer to the "get" part is user testing (you already answered the "manage" part)
–
NirOct 19 '10 at 10:55

1

+1 though the problems of a distributed userbase and localization (which may be community-generated) make hallway testing unrealistic. The application itself will include multivariate testing capabilities, though I honestly hadn't put too much thought into a multivariate testing tool which runs multivariate tests on its own interface - "head you like A/B testing so we put an A/B test in your A/B tester so you can test while you test" - I like it :)
–
danlefreeOct 24 '10 at 6:14

+1 for clicktale - I think there could definitely be some potential there, though I'll consider clicktale for the demo (versus the distributable code) to avoid any potential privacy concerns.
–
danlefreeOct 24 '10 at 6:16

This question also addresses a larger issue that FOSS projects, and all corporations face: how to weed out bad suggestions and ill-informed UI opinions from truly useful ones. Design documentation and previous Q&A you can point to (like idea torrent) is your first line of defense. Bill Fulton works on this a lot, I touched on it in this blog post (he doesn't publish much)