Beta testing and beyond

Ravelry is still officially in “beta”. This word has been drained of most of its meaning in today’s web but here is what it means to me (and hopefully our users): the site is being very actively developed and things will change, unfinished “trial” features will be introduced, there will be bugs. In the last 365 days, there have been 241 releases where a new version of the code is pushed out to the production site.

As an aside – I’m looking into doing away with the beta label and having beta testing being an opt-in thing for users who want to be guinea pigs. We’re approaching 150,000 registered users and not everyone realizes that they often part of an experiment – this ends up creating more work for us in the form of questions (and sometimes, complaints)

Our users have been incredible – they report bugs of all kinds, come up with tons of great ideas and suggestions, enter things into our issue tracking system, and comment on existing issue. Here is how we do it:

Collecting feedback

Some sites collect feedback from beta testers with an email form. Don’t do this. All of your users need to be able to see and read the bug reports and ideas that everyone else is sending in. Working as a group is more efficient (less duplication), more productive (because small what-ifs can turn into great ideas through conversation) and more fun.

We set up one board in our forums that is specifically for talking about Ravelry itself. The board isn’t limited to feature requests and bug reports – it is a place for any kind of Ravelry talk like “Who has the biggest stash on Ravelry?”. In the last 30 days, 2300 people have posted on this board and 18000 people have read this board. hm… I guess I have to come up with some ideas to drum up participation – people who aren’t reading are missing out on their chance to shape the site.

Keeping Track

Our Ravelry discussion board contains 97,000 posts. We can’t use the board itself to help us track, categorize, and update all of the bug reports and suggestions. Even if we tried to do it by keeping the threads themselves excessively organized and moderated, we’d be sure to lose things.

So… we have an issue tracking system (The List) built in to Ravelry. Jess, Mary-Heather, myself, and 30ish other Ravelry users called “trackers” have the ability to turn posts in the forums into new issues in our system. When people post bug reports or suggestions we all try to either add them to The List or connect them up to existing entries. You can see that the post below has 1 agree vote and 1 disagree – sometimes the agree/disagree votes that are part of our forums can be helpful when we are looking at suggestions.

Now what?

When I am not actively working on them, the list items serve several purposes:

Users can search The List to see if their suggestion or bug report has already been brought up

Trackers can connect future forum posts to list items in case people have more or different information to share. This happens a lot and it is handy to have slightly different bug reports attached to bugs and further thoughts attached to suggestions.

All users can comment on items and I read these comments when I set out to work on something.

Although I do have some basic prioritization type features to The List itself, I do all of my work-gathering and organizing outside of Ravelry. I tend to go into the list, gather a set of items that fit with what I am working on, and take those back to my virtual work area so that I can sort through them, read comments, and organize.

I’m really happy with this. I enjoy talking with Ravelers about the site and getting ideas and help from them. I’m also very glad that I don’t have to worry as much about losing good ideas and bug reports in a sea of posts. Ravelry Users: You can find the For the Love of Ravelry (FTLoR) board and a link to The List on the forums tab. As always – if you have ideas on how we can improve tracking and The List, we’d love to hear them. Just make sure to post them in FTLoR so that we can track them

Comments closed | Trackbacks closed

Comments (21)

On Ravelry, including FTLoR, I tend to read and often agree, occasionally disagree, but I virtually never chime in (unless something comes up that I vehemently disagree with! which is rare), because I feel like you and your partners in crime have so much on your plate already. So I feel like asking for more or better is — greedy! Since there are already so many planned enhancements as well as things we don’t even know about happening constantly.

But if you are actively soliciting comments on The List, feedback, other ideas — you got it! I really like the way you have the feedback loop, if you will, set up, and I agree that many minds are more powerful and creative than any one alone, as far as brainstorming. Plus one person’s strong opinion won’t drown others out in this exchange of ideas. (Well, unless it’s yours, I suppose, wink. Just kidding!)

Thanks for all you do; again, most interesting behind-the-scenes look.

Ravelsourcing – is there any need for codemonkey interns/assistants? I am an Oracle DBA by trade – but am dipping my fingers into mysql. I’d be happy to volunteer. (However, I know Ravelry is your intellectual property, so I understand if you decline.)

It’s all google’s fault! Gmail is still beta after over four years. People tend to forgot what beta really means. I’ll be opting for the beta option (hopefully I’ll have the choice). You’ve done a great job! Thanks!

Thanks for the extended info Casey… I often wonder how the wheels are greased and turned on Ravelry… You have a great network here and I love all the tools you’ve added to the site… For certain, I’d opt-in for Beta! Thanks again!

I read separately that there might be a guinea-pig opt-in option on our profile pages, but if you’re tracking volunteers some other way, I officially volunteer to be a guinea pig. I would be able to give feedback on Windows, Mac, and iPhone OS’s.

Thank you, this was a very interesting read.
I will, too, be a beta tester if I have that option, being part of the growth of Ravelry is such a fun ride, I wouldn’t want to miss it. Also to be given the chance to help out, even just a tiny little bit, is great.

thank you and congratulations, once again, for all your hard work
Vero

This was very helpful… thanks.
Much like the first commenter I rarely bring up (minor) issues because I know you have plenty to do and you’re still working on things (I do understand what being part of beta means). But if you want us to speak up I will. And I never used the agree/disagree button but if you’re also tracking that then I surely will.
Ravelry is such a big place… sometimes it’s had to know where to go to post the questions and to find the answers. Thanks for the directional pointers.