Make a filter selection page the default entry point for the review queues

Show the size of the queues for tags they are likely able to work with - not the whole queue

When all review options are done, show the user the impact they have had on the view of the queue they were given to start with

Long version:

There are (at least) two things that discourage people from getting into the /review - the appearance of an endless list that one can't make a dent on and having to 'skip' all the time to get to questions that one does know how.

I'll also point out that I suspect that some honest review audit failures come from people trying to review things they don't have a grasp of. Making it easier for people to review what they do know may make it easier to honestly pass audits.

Under this model instead of showing someone trying to make a dent in "84,803 need review", break it down to the tags and reasons that a person can work with and see a reward from.

I'm going to use Robert Harvey for this because he has more interesting tags than I do. I'm also going to pretend that everything he answered or asked is in the close review queue for the purposes of data.

Take the top 50 tags a person has. This is about the first page of tags. The reason for this is that it helps ensure that there is enough to build up a queue. It also helps identify the questions that a person is likely able to cast the appropriate close vote (or leave open) on without guessing about technologies one is less familiar with.

Instead of clicking on the review and just getting at the queue with that little filter link at the top, one instead is taken to a filter selection page where one can build up the appropriate filter to work from.

You could select wedges from the pie chart, or select them from the list on the side. Furthermore, there would also be a place to enter other tags.

Additionally, there would be a breakdown by close reason that one can select to build up the appropriate filter.

With both tags and close reasons, a question can fall into multiple spots. For the purposes of this, show it in both tags or reasons.

Upon completing all the reviews for the day, show the information again, and allow them user to see the impact on the tags and reasons they chose. This is the reward - not a badge or something, but a visible impact on the size of their corner of the queue.

One of the things this may help do - by presenting this information - is hep engage the various tag communities around the site to go and work on the review queue for their tag. Seeing that some tag went from 3894 questions in the queue to 3563 in the last day may help.

Allowing a person to go and address just the off topic - recommend or off topic - migration (note, those are time sensitive) -- or even just 'off topic').

Of course there should still be a 'just take me to the queue' (and one can still set up the filter in the queue that way). The key is to show the impact of what they have done to the queue in a way that is meaningful and not overwhelming.

A mockup to give an idea of what I'm thinking about. Don't look too deeply into the numbers, they are more as an example.

You hit your standard 'review' link to get to the close review queue. Instead of taking you right to the reviews, you get a page of stats that lets you see what is in the queue and modify the filter.

Note the 'Other' box at the bottom which has the same functionality as the 'tag' input that one can get from the current filter link.

I should also point out that the 'Review!' link can be clicked on without setting up any filters if you want to just "take me to the queue" and tackle whatever is there or use the old filter style setup.

You go through, you do your reviews and then you have a page that shows the queue once you've exhausted your tag or your reviews for the day.

Thats really what its about. Make it easy to set up the filter of things you are likely able to review, make it easy to see what effect you have had on the queue when you are complete with the review.

Great idea, but 3 comments. (A) Save the pies for dessert.. Arranging with bar charts would be much nicer. (B) Arrange the close reasons by the order they appear in the close vote list. (C) allow filtering by nothing, by close reason, or by tag (or both) -- don't force people to click through (a handy menu on the side showing the last 10 review filters you made and what the number of pending items are would be nice)
–
jmac♦Nov 21 '13 at 23:42

1

@jmac In my defense, I was more going for a 'this is an idea...' not 'this is a display that it should be'. A tree map might be appropriate if one could visualize the tags in it, somehow... but then thats why I don't do UI. The key point is we need a way to both rapidly select filters and see the impact we had on the queue for the filters. If this is taken seriously (chicken & egg?) I would certainly put much more thought into the design (and maybe even a question or two on UX.SE).
–
MichaelTNov 22 '13 at 1:45

how do you expect this to work for small CV queues? Or this idea is targeted primarily at stackoverflow?
–
gnatNov 22 '13 at 13:55

4 Answers
4

This feature was implemented and went out in build rev 2014.5.16.2252 on meta and 2014.5.16.1610 on sites.

Currently it's enabled only on stackoverflow. Note, that we have a big threshold (50) for how many outstanding reviews there must be with a tag or close reason in order for it to show up on the dashboard. Also, as opposed to the mockup, we don't display counts, since those are very expensive if we'd want exact numbers for each user.

As of build rev 2014.5.19.1613 we don't show any of your ignored tags on the review dashboard.

When I use one of the links I cannot change the filter of the Close View. To be more specific, when clicking on duplicate and the adding the c#-Tag as filter the c#-Tag is not applied to the query. Is this by design?
–
JehofMay 19 '14 at 9:14

that's not a bad idea. maybe also bump favorite tags? i've never found the need to ignore tags
–
mhlesterMay 19 '14 at 15:27

@mhlester we discussed bumping favourite tags and decided that it was not a good idea since they may not be enough outstanding reviews. the are tags displayed are ranked on the global outstanding review count (not per user) base so you can quickly run into the scenario where the user has no more reviews for in a tag but would still see the link...
–
m0sa♦May 19 '14 at 15:33

I just saw @Shog's revision notes (couldn't pick it out on its own). That does an even better job than i'd hoped for!
–
mhlesterMay 19 '14 at 15:37

What exactly is enabled only on SO and what is enabled everywhere?
–
BraiamMay 19 '14 at 23:59

I like this idea, but I hate the notion of a click-through page, and I really hate those big honkin' pie charts.

David Fullerton proposed a fairly simple UI change that could make filters a good deal more obvious without unduly disrupting the UI:

Keeping those counts up-to-date is expensive, so we'd probably have to limit them to the broadest possible filters; still, this could go a long way toward introducing folks to the idea of filtering, as well as providing one-click access to each specific close reason.

Finally, I would like to bring this whole concept out of /review and drop it directly onto the individual tag pages. Imagine visiting java and seeing a little red indicator near the top calling out the number of pending review tasks for that tag... Clicking takes you directly into review.

Only 1 additional click to get to the queue (I see I need to mock it out a bit more) - "Of course there should still be a 'just take me to the queue' (and one can still set up the filter in the queue that way)." The key is to get people into doing the reviews. Many times I've seen comments to the effect of "you mean, you can filter the queue to just see off topic?" - they aren't the most obvious things to find or set up. This makes them, well, in your face at the start - to create and set up and see the effect upon when complete.
–
MichaelTOct 20 '13 at 10:05

While I've not ventured in the close vote queue, with my meagre 800rep, I imagine it doesn't matter what the tag is most of the time.

I've reviewed things in tags plenty where it doesn't matter what the tags are.

I hate everything and so am posting this funkyness cos I'm blur brained - wibble wibble I am a fish

I can easily review that, even if it was in the "rocket-science" tag... and I imagine similar scenarios are most common for the close vote queue too. ie it's clear cut.

Against
There's a potential that people will go to their favourite tag only.
Currently users are reviewing everything thrown at them, and so currently might do a few in certain tags which given the choice of specific review tags they would not.

If the same people are willing to do the same amount of work, but on specific tags only, then when those tag reviews are empty (for the smaller tags) they do nothing, whereas before they might have reviewed another tag.

Will people do more just because it's a tag they like? Does it really matter, as they're not answering questions on their favoured tag.

For
The flip side, is it might actually work better in terms of people doing more reviews than they currently do now "because" they can choose tags of their interest.
So the tags with the biggest percentage of close votes might actually get more done.

Obviously the main issue with the close vote queue is it's a never ending queue so people get bored. They do what they can, leave.
If they could do a load on their favourite tag, they might do more than they would have.

Especially as the queue would be seen to be deteriorating.
In fact, it could become a bit of a competition between tags as people try to get theirs down faster (I hate this thought, like being back at nursery, but it does happen/work)

And to contradict myself from the against [gracefully: as there is a valid point]:
If the same people are willing to do the same amount of work, but on specific tags only, some tags might get more action and thus people actually do more work as they enjoy reviewing their tag of choice more than just any thrown at them.

Although, I'm not convinced this would be the case on close vote queue, as even people loving certain tags might not make them more likely to do more close votes.

Conclusion

I'd say it'd be well worth trying, but it's likely a s*#t load of dev work for what is a case of testing the water. If you could give more conclusive reasoning as to why this would work, stronger theories, perhaps people saying here if they'd do more if it was a tag of their choice, it might be considered*

(*no, it wont... )

Everyone: If you would do more on the close queue if you could do chosen tags, upvote Kate Gregory's comment!

While there are definitely many questions that are obvious to see that they should be closed, there are also plenty that aren't obvious - many requiring domain knowledge to analyse properly.
–
DukelingOct 20 '13 at 2:42

4

I take umbrage to your phrase "I imagine it doesn't matter what the tag is most of the time". Context does matter, especially to tell the difference between a question that could use a little love, or one that's been asked for the umpteenth time.
–
MakotoOct 20 '13 at 3:46

Well, my main point is, implement this and people will stop doing all tags except theirs, so don't expect help on any others. It could be detrimental to the queues.
–
JamesOct 20 '13 at 12:50