Put an accepted answer into production code for a commercial website, which was then subjected to pre-production testing. A significant, easily-reproducible bug was found, tracked down and fixed. The fix was edited into the accepted solution, which was rejected during peer review. It seems unlikely that the peer review involved any testing of the validity of the edits. The reason given for rejection appears to have been purely pro-forma.

You seriously put untested sample code into production? Now I know there's no rule or such that prevents anyone from doing so, but, you know, we programmers have our own breed of what others call "common sense"...
–
BoltClock's a UnicornFeb 10 '12 at 15:02

Can you show a link to the answer in question?
–
PëkkaFeb 10 '12 at 15:03

@BoltClock'saUnicorn note OP wrote bug was discovered at "pre-production testing" not in production
–
gnatFeb 10 '12 at 15:10

3

@gnat: Makes this even more perplexing. How can production code be subjected to pre-production testing?
–
BoltClock's a UnicornFeb 10 '12 at 15:11

@BoltClock'saUnicorn agree the wording sounds slippery indeed. Since pre-prod testing is mentioned, most likely OP refers to "production candidate code" but I am not a mind reader to guarantee that
–
gnatFeb 10 '12 at 15:16

This really isn't that confusing...he's trying to emphasize that the bug was found by professional testers, and explaining why rigorous testing was involved by mentioning that the code will be going into production for a commercial website (obviously it hadn't yet -- most sites don't wait until after the code is actually on the production servers to do the testing)
–
Michael MrozekFeb 10 '12 at 15:26

9

I'll bite - I am pretty sure I voted to reject the edit - it amounted to a pretty cryptic screed in a comment inside a code block about how this answer was wrong + modifications to some of code. It looks to my eyes like an invalid edit. It should have been posted as a separate answer and a comment left on the offending answer pointing out what was perceived to be wrong with the answer.
–
talonmiesFeb 10 '12 at 15:52

3 Answers
3

Post your own complete solution pointing out the bug in the accepted answer, then post a comment on the accepted answer referring back to your corrected answer. If the correction is valid, then it should either be edited into the accepted answer, or the new answer should be accepted instead.

This is the correct course of action. Also be aware that answers aren't guaranteed to be bug-free - this is why comments and new answers are allowed. Peer review didn't necessarily fail in this case - any significant change to an accepted answer with lots of comments indicating that the answer as-is worked, suggests that changes should be made with care. In this particular case a comment or new answer was more appropriate.
–
Adam DavisFeb 10 '12 at 15:53

The fix was edited into the accepted solution, which was rejected during peer review. It seems unlikely that the peer review involved any testing of the validity of the edits. The reason given for rejection appears to have been purely pro-forma.

I saw, but did not take any action on, your second suggested edit. My opinion was "well, this might well all be true, but I don't think much of the tone. I'll let this be someone else's problem[1]".

As to what did eventually happen to your suggested edits, as the faq says,

What is reputation?

Reputation is a rough measurement of how much the community trusts
you; it is earned by convincing your peers that you know what you’re
talking about

When you come along with three days of membership and no more than the starting amount of rep, it's not particularly likely that peer review of your suggested edits will necessarily involve rigorous testing of the change; and (as might be learned from experience of suggesting edits), nearly all rejections of suggested edits will be with reasons that are "purely pro-forma", because the volume of suggested edits is such that it's impractical to do much more.

None of the answers here are offered with any warranty or guarantee as to fitness; therefore it's not really surprising that no major fire alarm was instituted when mistake was reported to have been discovered in one. There's no SLA here, or response time guarantees, or anything like that: just a bunch of volunteers who, like any group of humans, sometimes make mistakes.

I always reject edits that attempt to fix technical errors in the answer. If the answer includes code that doesn't check for something, and I see a suggested edit that adds the check, I reject the edit as

This edit is incorrect or an attempt to reply to or comment on the existing post.

Comment on the answer if you have the rep for it, or add your own answer if you don't. (Your own answer should not say just "the answer by Bob has a horrible flaw"; it should present the correct answer with the flaw fixed.) Edit-reviewers can't tell if you're right that it might leak memory or delete a good file or whatever the horrible problem is, so editing the accepted answer is not the right mechanism for fixing the Internet in this case.

Problem is nobody ever rigorously tested the original code. I did test it. Hopefully, the end goal is to produce correct answers, not to follow some proforma rules. An intelligent peer review, as practiced by academic journals for example, is based on the sense of the material being reviewed, not on abstract, automatic rules.
–
kcrossenFeb 10 '12 at 16:19

7

I am not arguing with your review. Only with your choice of "suggested edit" as the way to submit your results. Would you use a tweet to fix a scientific paper? And expect it to work?
–
Kate GregoryFeb 10 '12 at 16:25

6

@kcrossen: you are missing the point. Stackoverflow is the ultimate meritocracy - answers are posted and voted on according to their accuracy and efficacy. If you find an answer to be deficient vote it down. That is what voting is for. If you have what you believe is a better or more correct answer, post it as your own answer and let the meritocracy do its thing. That is what passes as peer review around here, not what you did.
–
talonmiesFeb 10 '12 at 16:28

1

@kcrossen in fact you did just what you say no-one did. You tested the code. Yay! We're all volunteers and you volunteered. I'm glad. Then (because our mechanisms are sometimes opaque) you didn't submit your results the right way. No harm done, you know the right way now. Please keep on reviewing and fixing.
–
Kate GregoryFeb 10 '12 at 16:34

1

I tried many different approaches (other than reading a LOT of faq, that is). I tried to comment on the original solution, not possible, and so on, you probably know better than I. So I asked what to do, got my answer, and amended my answer accordingly. Thanks.
–
kcrossenFeb 10 '12 at 16:42