Please post new bug reports, feature requests or discussions as individual questions tagged with at least review in addition to the requisite

And thanks for all the great feedback so far!

We are currently soliciting feedback for an improved version of /review to help with the very important task of maintaining site quality. There are three targeted tasks we have built thus far and will continue to roll out others after we gain confidence these three are working:

Low Quality Posts: This task targets posts that we've algorithmically determined to be low quality. Your job is to read these posts and:

decide if a post does in fact Look Good

Edit the post to make it better

Delete1 the post entirely

or skip it if you are Not Sure.

The minimum reputation to perform this task corresponds with the ability to edit any post (currently 2,000).

Suggested Edits: This task targets posts edited by users who have not yet earned the reputation to fully edit. Your job is to read these edits and:

decide to Approve the edit as is

Reject the edit

Edit the original post and make your own edit

Delete the post (moderator only)

or skip it if you are Not Sure.

The minimum reputation to perform this task corresponds with the ability to edit any post (currently 2,000).

Close Votes: This task targets posts that have received at least one close vote. These posts generally need additional review to help decide if a post does in fact require closure. Your job is to read these posts and:

decide to Close this post by casting another close vote

Do Not Close2 this post if do not agree with the reasons for closing

Edit this post to make it better

or skip it if you are Not Sure.

The minimum reputation to perform this task corresponds with the ability to cast close votes (currently 3,000).

Each of the actions on the review task listed above may not always be available based on your current reputation or moderator status. We will be reviewing the data collected in this experiment closely to determine if these new review tasks are working and also reserve the right to pull these tasks out entirely if we find they are not. We have enabled these review tasks on Meta, Super User, Server Fault, and Stack Overflow.

1 If you don't have enough rep to delete, you can "Recommend Deletion". This doesn't do anything right now except move it to the front of the queue for people who can delete it.2 "Do Not Close" just removes it from the queue right now -- it doesn't affect close votes.

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. More info: help center.

4

@nhahtdh: "These will soon be moved to the new beta Review Task system" - So yes.
–
animuson♦Jul 12 '12 at 17:58

4

What happens with "Not Sure?" What if you want to come back to it later? There doesn't seem to be a mechanism for un-dequeuing a post.
–
CodeGnomeJul 12 '12 at 18:40

@octopusgrabbus you are still making progress. We will add this back to the new review task soon.
–
Geoff Dalgas♦Jul 12 '12 at 20:27

9

[cross-posted from SF.chat] You need 2000 rep to use the first two links. I have 1.4k rep on SF with 400 helpful flags. I'm going to still have to use the "old" links until they get move or I get more rep. I'm not expecting for the whole system to be changed but could they be made avalible to those with 2000k rep or 250+ helpful flags? That number demonstatres a decent understand of the flagging system.
–
tombull89Jul 13 '12 at 7:38

1

Are there 85,3k CloseVotes or 85,3k questions with at least one close vote? The second metric would make more sense.
–
örsJul 13 '12 at 8:52

5

Isn't a required rep of 2000 a bit excessive. Its not like a 1000 rep user wouldn't be able to tell if a question was low quality or not.
–
VBwhatnowJul 13 '12 at 10:52

6

With such high rep limits, review might start stalling, especially on AskUbuntu, where panicky users are great in number, and returning users are few.
–
hexafractionJul 13 '12 at 12:28

5

Just want to say that I really like the direction the review workflow has headed. Usability while reviewing was probably my biggest (of few) gripes with SO, so I'm super-excited. A huge thank you to the team for this.
–
Rob HruskaJul 13 '12 at 16:37

16

The "not sure" option rubs me the wrong way. Makes me doubt myself sometimes when all I want to do is move on. Couldn't it say "Pass" or "Skip" instead?
–
j08691Jul 13 '12 at 17:27

1

@PopularDemand - Yeah I had the same thought, although "not sure" connotes a meaning behind the action whereas "skip" or "pass" simply denote the action of moving onto the next review item.
–
j08691Jul 13 '12 at 18:44

4

I like it, great idea!!! One comment: the close vote tab should have a big warning: CAUTION, USING THIS FEATURE CAN REALLY RUIN YOUR DAY (and an ad of the leading antidepressants brand). It's really depressing to see the amount of crap coming by when doing it for half an hour. Worse than the "low quality posts". Ehh, I guess it works...
–
Gert ArnoldJul 15 '12 at 11:41

3

@Gert: that's not a terrible idea... I know I can't spend too much time wading through the 10K/mod tools without sinking into despair. Maybe when you run out of close votes it should automatically redirect you to stackoverflow.com/questions/greatest-hits... er...
–
Shog9♦Jul 16 '12 at 3:41

2

I'm a <2K user and I like the old system as it is. I couldn't access to any of these new sections, only to the statistics. Why? That would be my "don't like, stay with old". And of course, I couldn't test any new feature.
–
Shegit BrahmJul 17 '12 at 7:01

1

This way is far more effective and addictive than before. I find myself spending more time looking at the question, and spending more time than in the previous version!
–
C. RossJul 18 '12 at 13:26

Conversion to comment requires full community moderator status. This is one reason a flag option should be included.
–
DragonLord the FieryJul 13 '12 at 4:35

42

So add a Recommended Convert to comment for regular users and Convert to comment for full mods. Problem solved
–
Ivo FlipseJul 13 '12 at 8:42

5

Convert to Comment should be very rare. Yes, a lot of people leave comments as answers - but that doesn't make them good comments. I've also seen "not an answer" flags being used as an attempt to deny rep for answers that ARE ANSWERS, but which the flagger feels are unworthy for whatever reason (incorrect, too simple, link-only, etc) - moving a poor answer into a comment doesn't fix anything. I'm not completely against providing an option for this if it's really necessary, but so far all of the examples I've seen should've been left alone or deleted.
–
Shog9♦Jul 15 '12 at 1:59

4

@Shog9 Especially with trouble-shooting questions on SU, many "answers" are actually asking for more information. They'd otherwise be flagged not an answer and hopefully converted to a comment
–
Daniel BeckJul 15 '12 at 21:02

4

@Shog9 "Did you try to X your Y?" isn't an answer, but might be useful to keep as a comment for certain questions.
–
Daniel BeckJul 15 '12 at 21:08

@PopularDemand My thinking is that questions and answers have slightly different criteria for being "low quality". If you have to constantly switch criteria in your head, you work slower.
–
JimmyPenaJul 12 '12 at 20:40

7

I think it is better if there is some kind of visual indicator to indicate it is a question or answer for Low Quality category and for Suggested Edit category. Currently, we have to scroll down near the end of the post to see "answered XXX" or "asked XXX".
–
nhahtdhJul 13 '12 at 1:29

2

Another reason for this is that you cannot usefully review questions once you're out of close votes. Only reviewing answers works though, and this would be one possible way to get that.
–
Daniel BeckJul 15 '12 at 21:00

Voting Up or Down is Missing

Under the old system, you could vote posts up or down while reviewing them. Since voting is one of the metrics tracked for the Reviewer badge, why is this ability not present in the new review interface?

Can the confirmation for delete please be removed? If not, provide an option. It's the /review route. I know a lot will be up for deletion. Having a confirmation is really annoying. At least skip the confirmation for Moderators

It would be great if all answers were visible, below the question. Having that helps to know whether it should be moved to a comment or deleted altogether

After approving/rejecting a suggested edit, the other button ( approve/reject) still remains.Fixed

Users are locked out of reviewing Close Votes when they run out of close votes for the day.

Consequently, the Top Reviewers each day are the users with the greatest "do not close"-vote to "close"-vote ratio, because they are able to accumulate the most reviews before being locked out. These users also have an advantage in the all time Top Reviewers list -- I took an early lead, but lost it and continue to fall further behind because I cast fewer "do not close" votes than certain other users.

This current design will skew votes towards "do not close", or at least encourage users to select "not sure" instead of "close" to avoid getting locked out. Instead, "close", "do not close" and "edit" should all count towards a daily review limit.

(Why am I including "edit"? It's good to encourage edits, but once we include "do not close", having a larger number of "edit"s would be the only way to get an advantage and would be the new factor determining the daily Top Reviewers. This would lead to a lot of frivolous edits bumping questions that should actually be closed.)

After that, consider removing the daily Top Reviewers list. It will be a lot less interesting once everybody's tied at a shared cap.

"Not Sure" really shouldn't be counting as a review - if you can drive yourself to the top of the leaderboard by abstaining then that's a bug. But yeah, "Don't Close" should be limited also.
–
Shog9♦Jul 13 '12 at 15:28

1

@Shog9 You're right, it doesn't count as a review. I just meant that users shouldn't be encouraged to press it when they actually believe that "Close" is appropriate.
–
Jeremy BanksJul 13 '12 at 15:32

1

I'd be thrilled if this could be changed before my close:do not close ratio causes me to lose my (slightly unfairly-earned) position at the top of the leaderboard.
–
Jeremy BanksJul 14 '12 at 14:30

This was the first thing I noticed. I tend to limit my reviewing mostly to my areas of specific expertise, where I'm most likely to be able to locate duplicate questions, etc.
–
Michael BerkowskiJul 13 '12 at 0:34

1

This isn't so much a problem when reviewing for writing quality, but it's certainly a major issue when trying to determine whether a post is on-topic or not, and whether it should be closed or migrated. I want tags back too!
–
CodeGnomeJul 14 '12 at 14:03

Awesome, glad to see you guys aren't slacking. SO's a great site now and will get even better. I frequent the review tab a lot, but my rep's < 2000, so I can't access any of those tasks. Can you offer the beta only to those who can access it? Currently, I'm seeing this every time I click review.

I agree with @SomeKittens. And I guess I understand why you might want only established users to be able to edit posts. But as someone who constantly edits everything I read, I am going to feel soooo left out.
–
theJollySinJul 12 '12 at 20:01

I ran into a problem with this today. I noticed a poor edit suggestion and went to just improve the post and do it right. I went in, edited the post, submitted it, and I was moved on to the next suggestion. I went to the thread with the actual post (thanks to the browser history) to review the question and I noticed that my edits didn't take effect. I went in from the post itself (rather than the new review queue) and re-made my edits. I was given an error that the title already existed.

Apparently if there is an error making an edit in the queue it just swallows the error, displays nothing to the user, and moves on giving the impression that it worked even if it didn't.

A few unrelated comments:

I also miss not having the number of approve/reject votes displayed, or the reject reasons displayed, as on the old permanent edit suggestion page. It also doesn't list the approval/suggestion stats or recent approvals/suggestions (which is less useful than reject reasons, but still useful).

If you hit 'not sure' for an edit and then go back to it it says you've already voted and can't vote again. While I can understand approve/reject votes being locked, and it's nice to not have something I've "not sure-ed" show up again in the queue, I would like to change my vote. This was prompted by a mis-click when I meant to reject a suggested edit.

When you run out of votes it would be nice of the approve/reject buttons just went away, and there was some indication at the top of the page so that I don't spend time looking through questions that I can't vote on.

Navigating the close menu for dupes is annoying. It AJAX requests every time you proceed to the next stage for dupes, and the other close reasons involve two clicks and a wait. Not that big an issue--but it disrupts the flow of the thing, especially for dupes.

The feature request is as follows:

For all options but dupe: just select the close reason beforehand so we can click "close" and be done with it.

For dupe, open the close menu at the point where you choose which question to make it a duplicate of. Even better, do this:

Enable the "duplicate" buttons only if the user has looked at the respective tabs.

This might make sense; I think this suggestion has legs, and could be integrated into both the normal question UI and the /review UI. We're gonna defer this for the time being, but will consider it after the rest of /review is ironed out.
–
Shog9♦Jul 25 '12 at 16:43

2

Tweak suggestion to sub-FR: don't put the "Duplicate 1," "Duplicate 2," &c. buttons on the header bar. Instead, put them inside the duplicate tabs, in the sidebar underneath "link." That will reduce the number of clicks needed to agree with an existing suggested dupe from a whopping four down to one, encourage people to actually verify that dupe suggestions are correct and prevent confused users from agreeing with dupe suggestions blindly. Also, the way the mockup is drawn, the "Duplicate 1" and "Duplicate 2" buttons kick the "Close" button out of its usual spot, which could be confusing.
–
Pops♦Jul 27 '12 at 14:48

One thing I don't like about the new suggested edit list is that it's no longer a list.

It used to show a list of up to 30 suggested edits, which made it easy to quickly scroll through them looking for things I actually know something about.

The new list shows me just one suggested edit, and I have to explicitly skip it (by clicking "Not Sure") before I can see another. It makes reviewing a lot of suggested edits much more awkward.

An unrelated thought: Should there be an option to show suggested edits tagged with my favorite tags, or to push them to the top of the list? Those are the edits I'm much more likely to be able to judge.Posted separately here.

+100. I really miss the list view. It made it much easier to locate the questions I am not qualified to review and skip the rest. Having to review the items one by one is much more time consuming.
–
LeighJul 16 '12 at 22:20

1

So far it still feels a bit tedious and awkward to me :/ Having to navigate through every item is much more time consuming than skimming a list IMO.
–
LeighJul 24 '12 at 22:09

What about the possibility of waiving the 2k reputation requirement for users who have earned the Marshal and/or Reviewer badge? I realize this is a small percentage of users, but I'd hate to see reviewing be taken away from those of us who have put in a lot of time on the old reviewing system, and who are still working to achieve the 2k goal.

If I want to pass to the next post is not only because I am not sure, but also, e.g., because I am reviewing low quality posts, and I am shown a question that I would vote to close, but I cannot because I have already voted to close all the questions I could.

Proposed Edits on Accepted Answers

It should be much more obvious (color coding? different border? clown cars and sirens?) when a proposed edit modifies an accepted answer. The tiny "accepted yes" on the right-hand side doesn't really leap out; you have to make a conscious decision to check for it each time.

There are actions I want to take on some posts but can't within the review interface, such as flagging or adding a comment (though this is allowed for low-quality posts), and I end up having to open the post separately to do so. Can you add more choices so that this isn't necessary? For example, old questions that should be closed but may not get enough close votes or should be migrated to a site not listed in the close dialog may be better handled by adding a moderator flag.

In a lot of cases I get an answer that's recommended for deletion, but often the question (and any other answers are crappy too). At the moment we lack a lot of context, because we can only see one answer and the question it was given on.

So I'd like an easier way to simply delete or close the question too, without having to open it on a new tab, delete the question, then go back and delete the answer (even though its already deleted).

It might be useful to add a check to see if any other posts on the same question are somewhere in the queue and if so, add them to the screen like the question is at the bottom.

(I had posted this as an addendum to another answer, but I thought it should be separate.)

I'd like to have an option to show suggested edits tagged with my favorite tags, or to push them to the top of the list (or, if there is no list, to give them some preference). Those are the edits I'm much more likely to be able to judge.

The interface lagged while I was trying to submit an edit from the Close Votes review page. The submit button appeared to be disabled after the first submission, but I think I pressed enter multiple times.

It looks like the client let me send the same edit multiple times, and the server accepted them all without collapsing it to a single edit:

In the Close Votes queue, I would really appreciate the ability to see the answers that have been posted to a question without having to click through to the question. A link ("View Answers") that would unfold the answers below the question would be ideal.

Like others here, I don't always have total confidence in my ability to identify duplicate questions about unfamiliar topics. On Stack Overflow, sometimes a question will concern an error that could have several different explanations, and one needs a pretty detailed knowledge of the topic in question to tell which explanation is correct and solves the problem. In that case, two questions could look like duplicates, but actually have completely different solutions, and a close vote would not be appropriate.

However, if the two questions have also generated very similar answers, and there's some indication that those answers were correct, then the chance of erroneous closure is lower and a close vote is warranted.

As an aside, I'd second other calls for the ability to filter by (at a minimum) tag and close vote type. With that in place, I think this will be a vastly more useful interface for reviewing questions with close votes/flags than the old flag interface. Thank you all for working on this!

I'd almost say that I just want the review page to render the whole question page and simply highlight the offending post. At least that way I get to see the whole picture without needless navigation
–
Ivo FlipseJul 13 '12 at 11:22

1

They could change the sorting to have the relevant post(s) be at the top of the answers, that would just require a custom sorting
–
Ivo FlipseJul 13 '12 at 12:04

This new system has many advantages over the old one, I believe that the only thing I really (but really!) miss is the ability to sort by tag, and some sort of relevance.

A couple of observations, though:

In the low-quality posts section, when presented with a short answer, the problem may not so much be the answer, but the question. It would be useful to have a simple way to work on the question, for instance voting to close it.

I was presented with something like 75% (if not more) of really old answers to equally old questions, where most of them already had accepted answers. Do we really have to wade through tens of thousands of old answers that the community has already relegated to the bottom of the pages by not voting for them? I am all for cleaning up stuff, but this is a bit too much even for me.

A suggestion could be to order the answers presented for review so answers with no votes and for which there is another, accepted answer are presented last in the queue. Especially if there has been no activity on the question for several months.

Similarly, in the close votes section, I was presented with a long series of questions on subjects that I didn't really feel an expert in (the lack of tag filtering again) and which were quite old (nearly a year). Feels a bit strange to be asked to spend time and effort on so many old and forgotten (many didn't show any activity for the last 10 months or so) questions where I don't really feel I have the insight to, for instance, remove a close vote.

Being able to up- and down vote these old questions would be a nice compromise. I'd feel more at ease just voting for a question in a field where I don't feel expert enough to decide whether it should be closed or not.

Lastly, the wording of the close reasons is slightly confusing, if not downright off-putting. Often a question is just a bad question according to the site's standards and expectations (that questions should show some self effort, be researched, etc.). In several occasions "Not a real question" is not correct - they are real questions, just not good ones.

In the case of a question that has lingered for months, asked by a user with a rep in the single digits, even if one leaves a polite comment, chances are that nothing will happen. There should be a simple and accepted way to close these questions for what they are, not for some bogus reason (too localized, narq, etc.).

If you don't have enough rep, the page just kind of...doesn't work. It's not immediately clear why I can't perform any of these tasks.

As a 500 Stack Overflow rep user, I see this:

Ideally, there could at least be something explaining the requirements (and that I don't meet them) for each task I don't qualify for.

So apparently it does say you need at least 2k reputation to review suggested edits. Maybe make that text red (when I don't have enough rep) to draw attention. I literally did not see that when looking over the page.

The message "You need at least 2k reputation to review Low Quality Posts" is only displayed if you're being locked out for that reason. However, this isn't clear; it just looks like it's a standard part of the description. It would help if the text were emphasized by being red or bold or something.
–
Jeremy BanksJul 13 '12 at 13:32

1

@JeremyBanks hah, I didn't even notice that at all. Maybe make the text red.
–
Ben BrockaJul 13 '12 at 13:36

3

The 2k requirement to ACT ON (detected) low quality posts is reasonable. But, it was one of the things that I used to browse periodically, both to flag non-answers as non-answers, but also to make edit suggestions of things I knew I was capable of describing more accurately or that I knew I could state more clearly what the person was trying to say. Is it possible that the "old" review system remain present for those users (like me) that can make use of just browsing the "low quality" questions/answers?
–
killermistJul 18 '12 at 20:38

When a question has a close vote on it, the duplicate is already closed and you vote to close as a duplicate as is suggested by /review you get this error:

As is the case with this question the most likely offender is going to be in the form of migrated cross-site duplicates getting closed out and people vote to close different questions as a duplicate with one "winning" the close and the other being left with close votes.

Under the "Close vote" section, it would be nice to have a smaller "reason selection" dialog pop up closer to the top of the screen instead of having the normal VtC dialog appear when Close is selected. Having to respond to the normal one interrupts the workflow.

It might be nice to do away with the normal VtC dialog all together when reviewers were simply assenting to the already selected close reason, but not making it so convenient that it was possible to accidentally click through.

Our philosophy with the design of the review UI has been to keep it fairly simple and avoid unnecessarily prejudicing you in favor of a particular action. Showing the existing close reasons (yes, there can be multiple) already heavily suggests that the question should be closed for one of them, but forcing you to select from the full list reinforces your role in deciding if and how the questions should be closed. That said, for extra-complicated decisions like duplicate-closing, there's definitely room for improvement; see also: meta.stackexchange.com/questions/138316
–
Shog9♦Jul 25 '12 at 16:47

Same goes for "close" in low quality once you've ran out of close votes.
–
slhckJul 13 '12 at 15:02

2

@slhck Questions shouldn't be shown anymore if you've run out of close votes; or the Close should become a flagging dialog (doesn't belong here) like for users below 3000 rep...
–
Daniel BeckJul 15 '12 at 21:19

1

This has been fixed and deployed to production.
–
Geoff Dalgas♦Jul 19 '12 at 17:55

I like this so far. However, while looking at the Low quality posts section, I had issues with it. The only options to choose from are Looks good, Edit, Recommend deletion, and Not sure.

What does Recommend Deletion do? Does it make a flag or something? I wouldn't mind having the flagging option somewhere on this page, I feel it's sometimes necessary. I may want to flag it as "Not an answer" sometimes, or point out advertisement.