We're trying to get an RSS feed of posts deleted with autoflags for chat consumption, but chat RSS is fraught with peril.

TL;DR:Charcoal is the organisation behind SmokeDetector. Since January 2017, we've been casting up to 3 flags automatically on posts that our systems are confident are spam. We'd like to increase that to 5 automatic flags on posts we're even more certain of to reduce the time spam spends alive on the network.

Who are you?

Charcoal is a user-run organisation that is primarily responsible for the spam-detecting bot, SmokeDetector. Over the past four years, with the aid of SmokeDetector, we've looked for spam on the Stack Exchange network to flag and destroy manually. In January 2017, with the blessing of Stack Exchange, we started running an "autoflagging" project, wherein our systems automatically cast up to three flags on a post if they're confident that it's spam. If you missed that happening entirely, we wrote a meta post on Meta Stack Exchange - or there's a slightly more concise explanation on our website.

How's that been going for you?

Good. We currently have 215 users who have opted into the autoflagging system (you can sign up too, if you're interested). We've flagged around 30 000 (29 592) posts, of which the vast majority (29 526) were confirmed spam - that's 99.7% accurate.

What are you proposing?

We'd like to expand our autoflagging system. At present we cast up to 3 automated flags on posts we're confident are spam; we'd like to increase that number to 5 flags on a subset of those autoflagged posts where we're even more certain they're spam.

Why?

Just so we're up-front about this: this is an experiment. Ultimately, we're trying to do these things:

Reduce the time that spam spends on the sites before being deleted;

Lower the number of humans who involuntarily have to see or interact with spam.

Increasing the number of flags we cast automatically on spam should accomplish both of these things:

Automatic flags are near-instant; manual flags take multiple minutes to be cast - that means that increasing the ratio of automatic to manual flags results in a shorter time before 6 flags accumulate and the spam is deleted.

Automatic flags are not cast by a human. Fewer humans, therefore, are forced to see/interact with the spam.

The data we have backs this up. In terms of time to deletion, we saw a significant drop in the time it took to delete spam when we started our autoflagging project. Take a look at this graph from the meta post on the subject for an excellent visual representation of that. Before we started autoflagging, spam spent an average of 56 hours per day alive across the network; with autoflagging in place, the average is much less, at around 7 hours per day.

What would this change mean for sites?

If this change goes ahead, these things are likely to happen:

It will only take 1 or 2 manual flags from users to spam-nuke an autoflagged post, instead of the current 3. Posts that are not autoflagged will, of course, still require 6 flags to nuke.

There may be an increase in posts spam-nuked entirely by Charcoal members, who may or may not be active on the site.

You will see a reduction in the time spam spends on the site before being deleted.

Fewer humans will have to involuntarily see each spam post.

The last two of those are indisputably good things. The first two, however, are more controversial, and are the reason we want to have a discussion here on meta before we make this happen. What follows are the major concerns we've seen, and what we can do about them or why we don't think they're an issue - we'd like to hear your thoughts.

The major thing we're looking for out of this is a reduction in time to deletion. The following graph shows how long spam currently spends alive on the top few sites; we're hoping to see a moderate reduction in the average times, and a significant reduction in the top outliers.

The following graph is from an experiment we've been running over the past week, casting between 1 and 5 flags randomly on each post matching the settings we're considering.

PostCount is the sample size; FlagCount the number of flags cast on each post in the sample; ATTD the average time to deletion, and CommonMax is the maximum of a 95% confidence interval. The major takeaway from these stats is that we're likely to see a ~5x drop in the average time to deletion, and a ~10x drop in the outliers.

Accuracy & false positives

Spam flags are a powerful feature that need some care in applying correctly. This is a concern that came up when we originally built the autoflagging system, so we already have safeguards built in.

We only flag a post if we're more than 99.75% sure it's spam. (Technically, the precise certainty varies by conditions set by the users whose accounts we use, but it's always above 99.75% - more detail on that on our website).

If the system breaks down or bugs out and starts flagging things it shouldn't, all Charcoal members and all network moderators have access to a command that immediately halts all flagging activity and requires intervention from a system administrator to re-enable. Outside of testing, that kill-switch has never had to be used.

We never unilaterally nuke a post. There are currently 3 manual flags required in addition to the automatic flags to nuke a post; this increase proposal still retains at least one manual flag.

We also make sure that everything has human oversight at all times. While only 3 humans currently have to manually flag the post, there are always more users than that reviewing the system's decisions and classifications; if a post is flagged that shouldn't have been, we are alerted and can alert the relevant moderators to resolve the issue. Again, this is very rare: over the past year, we've flagged 66 posts that shouldn't have been, compared to 29 592 spam posts (that's 99.7% accurate overall). We allow users to set their own flagging conditions, provided they don't go below our baseline 99.75% certainty. We recommend, however, a higher value that has a certainty of 100.00% - those who set their conditions below that are likely to see more false positives flagged using their account.

This proposal decreases the required manual involvement to nuke a post; to compensate for that lower human-involvement barrier, we will correspondingly increase the required accuracy before casting the extra automatic flags. For example, we currently require 99.75% accuracy before casting autoflags; we could require 99.9% accuracy for 4 autoflags, and 99.99% accuracy for 5 autoflags. (For reference, humans are accurate 95.4% of the time, or 87.3% on Stack Overflow - those are stats that jmac (a former Community Manager) looked up for us last year when we started autoflagging).

In the rare event of a legitimate post getting autoflagged, we also have systems in place to ensure it isn't accidentally deleted and forgotten about. Multiple people review each post we catch, whether it's autoflagged or not, and classify it as spam or not; if an autoflagged post is classified as not-spam, the system posts an alert to chat to let us know. That lets us ping the necessary people to retract their flags, and keep an eye on the post to make sure it doesn't get deleted.

To make it starkly clear how accurate this could be, here's a visualisation:

That's a chronological representation (left-right, top-bottom) of every post that would have been flagged under the settings we're considering for 5 flags, and whether they were spam (green squares) or legitimate (red squares).

Community agency & involvement

As I said earlier, this proposal reduces the required manual involvement to nuke a post. Since Charcoal members also cast manual flags on top of the automatic flags cast by the system, that's also likely to increase the number of posts that are nuked entirely by Charcoal members, without involvement from users who are active on this site. Some posts already have 6 flags cast on them by Charcoal (including autoflagging and manual flags), but the proportion of posts that applies to is likely to increase.

We don't think this is an issue in terms of subject matter expertise: the spam we see on the Stack Exchange network is broadly the same wherever you go - you don't need any subject matter expertise or activity on a particular site to be able to tell what's spam and what's not. We do, however, recognise that it's possible that a site's community may want to handle its own spam; if that's the case, we're happy to turn the autoflagging system off on that site or to retain it at its current levels. Just post something on your site's meta and link us to it.

What now?

We want to increase the number of automated flags from 3 to 5 to reduce the time spam spends alive on the network. We'd like to hear your thoughts. We appreciate that quite a lot of the stuff we do at Charcoal is fairly invisible to the sites, so we want to be as open as possible. If you'd like data or specific reports, let us know and we'll try to add them in - we already have a lot of reporting around autoflagging, so it may already exist. If there's other stuff we can do to explain or to help you make an informed decision about whether you want this, drop an answer or a comment on this post. Charcoal members will be hanging around this post to respond to your concerns, or you can also visit us in chat.

What about sites that only require three flags to nuke a post? Will that be increased to two autoflags?
– Sonic the Stay-Home HedgehogMar 5 '18 at 4:52

5

@Ano They're excluded from the experiment above. If we go to 5, I would expect those two sites to go to 2 flags (one manual to nuke, just like the 5 on a 6-flag site).
– UndoMar 5 '18 at 4:53

5

@Ano 'Excluded from the experiment' means 'we kept the status quo' - the status quo for the 3-flag sites is currently a single flag.
– UndoMar 5 '18 at 5:12

43

There are only two "three flag sites" and there will probably be zero in the near future. Let's not get sidetracked.
– Shog9Mar 5 '18 at 5:14

4

@Ano There was a big wave on Workplace and ELU a while back; three flags was meant to be a stopgap and just... never got changed. It's not really worth discussing in depth here; those sites are a small portion of the network in any statistic you look at.
– UndoMar 5 '18 at 5:19

5

Sounds like this system should be put on the front end, checking posts before they are even published, instead of having to wait for individual users to raise flags (even automatically).
– jscsMar 5 '18 at 13:22

5

We'd like to, @JoshCaswell, but that requires integration with SE and SE dev time, which is short right now.
– ArtOfCodeMar 5 '18 at 13:25

As an alternate solution to this problem, could StackExchange implement a system that hides posts with multiple spam flags from the site until they are dealt with in a review queue (one way or the other)? That would accomplish "You will see a reduction in the time spam spends on the site before being deleted." and "Fewer humans will have to involuntarily see each spam post." without reducing human oversight on the flags.
– Ajedi32Mar 5 '18 at 17:06

7

Do you have any statistics on how many page views happen during that reduction from 50 to 10 second reduction in time? I don't care about 40 seconds difference unless you can prove that a significant number of people see the spam during that time.
– Cody PMar 5 '18 at 19:17

4

@CodyP Ultimately, each spam post takes six flags to delete (ignoring moderators for a moment). Dropping the number of human eyes that have to look at this stuff from 3 to 1 is a huge benefit IMO. Some of it is nasty, nasty stuff designed to haunt the mind - at a minimum, we're cutting the number of people who have to view it to destroy it by that much.
– UndoMar 5 '18 at 21:06

6

@sampablokuper How we've used the word "nuke" here is in line with how the rest of SE uses it. I appreciate it's possible to not have come across that usage before, but that doesn't make it invalid - it simply has a slightly different meaning on SE than you're used to.
– ArtOfCodeMar 5 '18 at 23:37

19

I just want to say, as much as I use Stack Overflow, I never see any spam on it, and that's a real testament to how well the system already works. Good job.
– Lightness Races in OrbitMar 6 '18 at 15:09

14 Answers
14

I see a very big difference between moving to 4 or to 5 automatic flags. I wouldn't have any issue with 4 flags, but 5 flags means that only a single user is required to nuke the post. This would probably still be safe if that single user organically stumbles upon the spam post, in this case the odds of a user falsely flagging a false positive are very low. But that isn't the case I'd be worried about. As far as I understand, you keep links to auto-flagged posts in various places, which is pretty much necessary for this system to be administered. A single overzealous user monitoring flagged posts and adding their own would be rather dangerous, if they are careless.

The odds of actually getting a non-spam post nuked are incredibly low if you assume that all flags are independent. But this assumption isn't necessarily true if you have a list of possible spam posts that users use to find posts to flag.

I also see seriously diminishing returns here. Making sure that most spam is removed in a few minutes is a worthy goal. I'm not convinced pushing it below a minute is worth additional risk.

One other concern of mine would be how Charcoal handles the far more murky area of self-promotion. My understanding is that there aren't any automatic flags that target posts that are likely to be self-promotion, but not blatant spam. But this is an area that confuses real users as well, so it would be particularly dangerous in combination with the 5 flag option.

In general, I like the project and do support automatically flagging posts on all sites. My major concern is that it isn't transparent at all unless you know it exists. You're using real user account to flag, a moderator that doesn't follow the main meta might never know there are any automatic spam flags, as they only ever see real users. This is not a new concern, but I think transparency is even more important if you increase the automatic flagging in this way.

The graph in the post illustrates the accuracy of the automatic part of 5 flags. As for the human element on that last flag, you're right that it's not necessarily independent. However, while some people are more zealous than others in flagging, multiple (significant, not just 2) people look at each post, and any disagreement will start raising alerts. That means it would take probably 4 or 5 people agreeing the post was spam for a false last flag to go unnoticed, and even then I might still pick it up when I read the day's transcripts.
– ArtOfCodeMar 5 '18 at 9:05

3

Meanwhile, the transparency issue is something we're always looking to improve where possible. We'd like to hear it if you've got suggestions on that front (that goes for anyone, not just @Mad).
– ArtOfCodeMar 5 '18 at 9:08

36

@ArtOfCode this puts the review of potentially bad spam flags entirely on the Charcoal team, the individual site communites and mods can't review this case (unless they know about Charcoal and monitor the site actively). I'm slightly uncomfortable with that.
– Mad ScientistMar 5 '18 at 9:08

3

I can understand not being entirely comfortable with it. Let us know if we can alleviate that, of course. FWIW, I don't believe I've ever seen a bad spam flag that's gone unnoticed by the Charcoal team. That's anecdotal, of course, not hard evidence.
– ArtOfCodeMar 5 '18 at 9:10

12

Oh - and on the self-promotion thing: you're right, autoflags stay away from that. It's too much of a varied issue for us to apply a network-wide filter to.
– ArtOfCodeMar 5 '18 at 9:10

16

The best way to improve transparency would be to have a dedicated account (or pseudo-account or five) to do the flagging, but that would require dev team assistance (and the discussion of its merits should go to Petter's answer)
– John DvorakMar 5 '18 at 9:54

15

@Undo I don't see any solution except dedicated bot accounts with a link to an explanation in their profile. Everything else would rely on moderators knowing about Charcoal in the first place, and knowing where exactly to look. Dedicated accounts would allow moderators to review the flags with the existing tools, and it would make it clear to them that they are automated.
– Mad ScientistMar 5 '18 at 15:55

1

Let's explore that a bit @MadScientist. I'm not promising this, but it's an option: What if each post got one flag from the SmokeDetector account (with relevant profile links), and the rest done the same way as now? I think that'd solve your concern without burdening us with creating 5-10 accounts and getting rep.
– UndoMar 5 '18 at 15:58

4

@Undo ideally said bot account would be generated by the grace of CMs, complete with unlimited flags and the ability to flag the same post multiple times.
– John DvorakMar 5 '18 at 16:08

1

@Undo there's still be a bit of potential confusion for moderators that don't know how it works, but it would solve almost all of my transparency concerns.
– Mad ScientistMar 5 '18 at 16:40

1

@ArtOfCode About the self-promotion. In Bioinformatics we recently had problems when a new user posted an answer with his own software and a user came from the chat with some aggressive wording . See this chat. It has triggered a discussion on the meta site to change the restrictions. But I would advocate to let the beta site on their own.
– llrsMar 5 '18 at 16:50

5

@Llopis, We've discussed that situation with a local moderator. The solution we've agreed on is to refrain from posting comments indicating how self promotion can be perceived.
– AndyMar 5 '18 at 16:59

1

What is we asked autoflagging users to put something in their bio about it? Not ideal but might alleviate a bit in lieu of bot accounts.
– ɥʇǝSMar 6 '18 at 0:37

3

@Undo Yeah, I get the risk factor, but if the accurracy for these higher risk posts is 100% (so far) then the chances are extremely low that a legit post will get deleted. Since the rest of the posts will still get only 3 flags, right? That's just my 2 cents. I don't see that much risk given the reward :)
– alexr101Mar 6 '18 at 16:35

2

That's exactly how I see it, @alexr101. Yes, the rest of the posts would keep the current status quo.
– UndoMar 6 '18 at 16:36

The biggest advantage I see in how the Smoke Detector project has operated thus far is its transparency: one of the most frustrating aspects of using these sites is how difficult it can be to determine precisely what happened in a given scenario (much less why it happened) - even for moderators, there's so much data that's either unavailable or more often available but not clearly connected that making sense of the sequence of events that underpin a complaint can be a real chore. For ordinary users, deciphering something odd they encounter can require knowledge of hidden tools and a 4-year degree in Meta Minutiae.

But Smoke Detector? It's all right there, out in the open and heavily crosslinked. What got reported, why / who reported it, what the post said and who wrote it, when it was created, when it was deleted, who flagged it... A massive record of spam in all its gory detail, laid out for review by anyone with an interest in doing so.

Entrypoints

With that in mind, it might seem ironic that most of the concerns expressed here relate in some way to accountability. But it's not: for all the data (and powerful searching / filtering exposed in the web interface), Smoke Detector suffers from one of the same problems that afflicts so many otherwise-powerful systems on Stack Exchange - discoverability!

There's no obvious way for me to see (as a user or moderator browsing my site) what - if anything - has been identified, flagged or deleted by the Smoke Detector project.

If I do come across a spam-deleted post, there's no obvious indicator that Smoke Detector had any involvement in this. The most likely route for me to discover this would be to observe a comment from someone not normally active on my site, but this rare occurrence lends itself to observing only activity on borderline posts; my perception of both the scope and worth of the project is inevitably skewed.

Fortunately, I think these concerns can be addressed fairly easily. My initial recommendation would be to generate a monthly report for every site on which Smoke Detector has identified posts, summarizing statistics for the post(s) identified and linking to a search that provides the full list of posts for perusal by those who might be interested. Such a report could be posted on each meta site on a regular basis, thus providing both transparency and awareness of the project without intruding on the day-to-day operations of the site itself.

For moderators researching specific complaints, a userscript to look up SD data on any spam-deleted post should suffice to allay concerns about false-positives going unnoticed. This would potentially require a bit of extra support from the Smoke Detector project, but I don't think it would need much work.

Easy to do a userscript; meta posts could be automated from a template and posted from the bot itself every 3-6 months. I like it, thanks for weighing in.
– UndoMar 5 '18 at 22:59

We have the brand new site dashboard on Metasmoke that does exactly this, but some mods are (understandably) wary of relying on a third party tool.
– a spaghettoMar 5 '18 at 23:02

16

I'm not overly concerned about the 3rd-party aspect, @quartata - rather, there's nothing on any Stack Exchange site that would clue you into its existence if you didn't go out of your way to look for it. It's like the 10K tools on SO, except there isn't even a faint grey link in /review - awesome stuff, but nobody knows it exists.
– Shog9Mar 5 '18 at 23:03

1

Monthly report * 173 sites = quite a bit of undertaking, unless those posts are being generated by a bot (through an account that accepts pings, ideally)
– John DvorakMar 5 '18 at 23:04

3

@quartata The problem (to me at least) is less the third party nature and more the fact that I have to go check, rather than just being told. If that being told isn't a genuine SE review queue but whatever private message, e-mail, or just chat bot, that's still better than having me remember checking out the dashboard.
– Christian RauMar 5 '18 at 23:04

3

An alternate (but to my mind much more obnoxious) route here would be to raise an "other" flag on each autoflagged post with a SD link to the report, @Undo. I hate that idea because it'd be a massive pile of flags requiring no action in the vast, vast majority of cases, but for very quiet sites it might make more sense than a userscript.
– Shog9Mar 5 '18 at 23:05

Automating report meta posts every n months is pretty easy once you get to the technical details of it; totally happy to do that. Probably worth letting sites say they don't want the posts, though.
– ArtOfCodeMar 5 '18 at 23:06

1

Yanno? I'd actually be happier with at least one custom flag, especially if its sticky. And it feels like something worth testing
– Journeyman Geek♦Mar 5 '18 at 23:07

2

Perhaps there'd be a way to set up an annual meta post and each month a new answer is added? Some of the sites I use barely have 12 posts per year... just naturally.
– Catija♦Mar 5 '18 at 23:07

1

@JohnDvorak metasmoke can alert CHQ when it makes a post; we can watch it for pings. Not out of the question at all.
– ArtOfCodeMar 5 '18 at 23:07

7

There are some really quiet sites moderation-wise, @John. These flags might be all that keep some mod-teams from going stir crazy. The Diamond Madness, they call it - one day you stumble on a lonely forgotten cabin in the woods, perhaps to take shelter from the driving snow, and upon opening the door you see the desiccated corpse in the blue hat... bony fingers still clutching that last flag.
– Shog9Mar 5 '18 at 23:15

5

If users could provide freeform text entry alongside any type of flag, Smokey could submit autoflags with a freeform entry of "Smoke Detector autoflag (report link)", and submit non-automated flags made via manual review via "Smoke Detector flag, cast by a human during review (report link)". Not that I want to use this to plug a thing I suggested, but... this seems like an ideal & relevant time to mention that thing.
– doppelgreenerMar 5 '18 at 23:17

2

Might be possible to do a combo approach - 'other' flags on the smallest sites spam-wise, meta posts on the ones large enough to have meaningful data.
– UndoMar 5 '18 at 23:20

1

@Undo SD could always flag (1 of the flags) and then x site basis you decide if spam flag or other depending on moderator team decision on relative site. I know lot of config :(, but often it's the only way to keep all happy.
– Petter FribergMar 6 '18 at 7:36

The thing that concerns me about 5 auto-flags is that sometimes users are wrong and now it only takes one user to spam-delete a post. A statistic that would help me better evaluate this is: on each site, how many organic spam flags are dismissed or disputed? If even site users sometimes get it wrong -- and they do, particularly on questions where product links in answers are expected -- then allowing a single user, native or (especially) visiting from Charcoal, to cast binding spam flags gives me pause. But I have anecdotes, not data; can we get data? (It would have to come from SE, I know.)

I'm glad to see your team working on tools to help moderators monitor what's going on. I'm going to need push notifications, not just pull (the dashboard), which I understand is possible.

Given that the users from SD see the reports, it is pretty likely that the sixth flag will come from a Charcoal visitor as well. I wonder though if that's much different with the three human flags used right now.
– JADMar 5 '18 at 14:50

2

Please note that these automated flags are only cast on posts of which the system is (at least) 99.75% sure it's spam. This means that only the really obviously blatant spam even has a chance of having 1 "real" user cast a binding vote. (Just to prevent confusion)
– CerbrusMar 5 '18 at 14:51

@JAD that's a good point; it'd be interesting to know to what extent spam is already being flagged entirely by people who are not otherwise users on the site. (Not sure how to measure that -- users whose only participation is flagging vs users with other activity, maybe?)
– Monica CellioMar 5 '18 at 15:26

1

@Undo probably a feed into a chat room of posts that were deleted with Smokey multi-flags. Or that Smokey multi-flagged, I guess, since that's going to be nearly identical and might be easier to generate. I'm open to other implementations; I know you can feed notices to chat rooms now and that seems like it'd work for this purpose. Also, thanks to you and Cerbrus for clarifying the higher bar for this to happen at all; I missed that.
– Monica CellioMar 5 '18 at 15:31

1

@ArtOfCode is that data available per-site? Since I don't know how my sites compare to others, an aggregate number doesn't necessarily tell me what's happening on my sites.
– Monica CellioMar 5 '18 at 15:33

2

It's not at the moment, no. The 95.4% is a number we got from jmac last year; the 99.27% is a strawpoll of Charcoal who were present when I asked.
– ArtOfCodeMar 5 '18 at 15:38

3

That can be done. Could even build an RSS feed of flagged posts you could plumb into your email/chatroom/IRC/whatever - would that work? It'd be a few minutes slower than straight Smokey posts, but far more flexible.
– UndoMar 5 '18 at 15:54

1

That's... sub-optimal for moderation, @Magisch. We need something like our site dashboard, but pushable rather than "there it is go get it".
– ArtOfCodeMar 6 '18 at 7:58

2

@E.P. Thing is, human flaggers aren't entirely independent. It would be very nice statistically if they were, but that's not the case. Humans are more likely to spam flag a post if it's already been spam flagged or negatively received; particular with the intervention of Charcoal folks, that means that human flags are interdependent and not statistically distinct.
– ArtOfCodeMar 7 '18 at 11:38

1

@E.P. As to the error rate - honestly, we don't know. At the settings we're considering for five flags, there are no false positives at present out of 30k posts matched. So, at present, it's 100% accurate, but we don't know when a false positive might turn up. It could be one in five million; it could be less. It could be more.
– ArtOfCodeMar 7 '18 at 11:39

1

@E.P. I don't have hard data on it - you'd need to convince a CM to get that - but the premise for human flaggers being not-independent is simple: If a given flagger (or two) makes a mistake on a post, it stands to reason that there's something about that post that tends to trip up humans. That plays out often in Stack Overflow's flag queue: multiple bad spam flags, from users with no apparent connection to each other.
– UndoMar 7 '18 at 15:47

4

FWIW, I'm happy to provide data on the accuracy of spam flaggers if requested... But please make it a separate question so I don't have to post that in a comment. Tease: it's probably a lot worse than you think it is.
– Shog9Mar 7 '18 at 16:06

I dunno whether bad luck, or good that it occurred, this happened yesterday:

The red color marked user is a well known overzealous flagger. I've seen many of their spam flags being retracted just a few seconds after flagging and then using a NAA flag. The blue marked user is a highly respected member of Charcoal (or atleast, I respect them a lot), and who is quite good at taking decisions.

In this case, the overzealous user flagged the post almost immediately. A few seconds later, the trusted user naa- reviewed the smokey post. After that the first user retracted the flag immediately and added a naa flag.

This wouldn't be a problem usually, but it would make a difference if that user is given the task of being the 6th flagger on a post with 5 auto flags.

Some suggestions to prevent this:

Check if a user has lots of invalidated feedback and flag it from their account, so that you can be sure that they are not the 6th flagger.

Check if a user has lots of invalidated feedback and stop/scold/educate the user to be more careful.

Use just 4 instead of 5, but this has the disadvantage (from the data) that it doesn't help much. So I'm not much attached to the "use 4 instead of 5" statement.

You people are quite brilliant at this, and can figure out a better way. My only request is to somehow make sure that a overzealous users isn't the deciding factor, if you are flagging 5.

Hot-headed users could be filtered out empirically. Say, give a one-week cool-down if you mistakenly nuke a post with your one manual flag. If it recurs, make it a month. And by cool-down I mean flag ban ;)
– Andras DeakMar 6 '18 at 0:15

4

@AndrasDeak Well certain moderators on smaller sites do send a mod message to users who wrongly flag. But I think we shouldn't be going to that level. Charcoal must ensure that it doesn't go to such an extreme level. And no, mods can't add flag bans. If that was the case, I would have done that.
– Bhargav RaoMar 6 '18 at 0:16

Are you saying the overzealous problem flagger actually is a proper Charcoal user (upto the point of having their account registered for auto-flagging)? Maybe that says something too and ought to be investigated, rather than thinking how best to "cripple" them.
– Christian RauMar 6 '18 at 0:51

@ChristianRau Yep, they're a registered user. I've marked a few of their autoflags as helpful.
– Bhargav RaoMar 6 '18 at 0:52

7

If that is remotely a possibility to happen, that speaks pretty clearly for not giving individual Charcoal users unilateral spam-deletion powers.
– Christian RauMar 6 '18 at 0:53

2

Yep @ChristianRau, that's true. That's the point of this post. However, I'd not say "Charcoal users" collectively, but just the small bunch of them who are a bit trigger happy. Most of the Charcoal guys are quite proficient in handling spam (like I mentioned about the trusted user).
– Bhargav RaoMar 6 '18 at 0:56

While this does demonstrate that some people are overzealous, it does also speak well of our review process - even though an incorrect flag was cast, it was caught almost immediately by other Charcoal members. That's a good illustration of why a post isn't going to get deleted and forgotten about; if this happens (which is rare), then we'll know about it and be able to alert the necessary people.
– ArtOfCodeMar 6 '18 at 1:05

Yup @Art, it's because of the good people who review them properly that Charcoal is now famous. I'm just pointing out a very minor issue which I discovered accidentally. If we manage to fix these minor issues then we can certainly gain the trust of others who are a bit skeptical of the system. (I hope my post didn't come across as "STAP IT IMMEDIATELY", as that is not my intention)
– Bhargav RaoMar 6 '18 at 1:10

1

Not at all, @BhargavRao, just wanted to highlight that element of it. There are definitely ways we can put a clampdown on things like this happening.
– ArtOfCodeMar 6 '18 at 1:13

I'm not terribly excited about a chatroom gathering together to coordinate flags on content in order to automatically delete it

I realize that the main objective remove spam quickly is important but I think we that coordinate flags/actions in chatroom has a great responsibility to listen to moderators on specific site (they have been elected) and we need to assure the members who do not participate in chat/flagging are never targeted or gamified.

With that said, these are some ideas to leverage the great capability of SmokeDetector to find spam and keep community moderators involved.

Get moderators involved on each site, listen and let'em decide how many auto-flags you can cast, as a side note I would be ok that moderator directly cast flags if they think that it's the best solution, they will be responsabile since community has put their trust in them.

Instead of using other users "real" accounts, create SmokeDetector flag account eg. SD_FLAGGER1, this will both remove the idea of user participating to get "free" flags and badges (I know they get very few flags but just the idea taste bad for the community). More important, it will be clear to moderators who is flagging the post.

How would you propose we get the moderators of each of the 172 sites involved more than a main meta post?
– MithicalMar 5 '18 at 9:35

Of course. If individual sites have an issue with this, absolutely we'll work with them to find the best solution - whether that's off entirely, at current levels, or something else. The decision on that needs to be a community decision, though, not mod-only. The debate about bot accounts is ongoing.
– ArtOfCodeMar 5 '18 at 9:36

Re: other accounts - that's essentially sockpuppeting, no? Who owns these accounts? That's essentially casting a flag with a sock and then manually flagging with your main account, which is pretty much a no-no.
– MithicalMar 5 '18 at 9:36

3

@Mithrandir I can't understand your first comment? You will need to contact each moderator on each site (teacher lounge for example), regarding second comment is not 1 account (SD) casting 5 flags with other accounts?
– Petter FribergMar 5 '18 at 9:39

2

@PetterFriberg the difference is that in one case you have five people vouching for Smokey, in the other you have Smokey vouching for himself. Another option would be to have the devs vouch for Smokey by giving it a special privileged account, but that hasn't seemed necessary.
– John DvorakMar 5 '18 at 10:36

11

@JohnDvorak I prefer Smokey vouching for himself, since it's it casting flags, it's more clear/transparent for moderators and you avoid the free flag/free badge idea. You do not need dev support, just setup flagging accounts.
– Petter FribergMar 5 '18 at 10:45

5

@PetterFriberg: each account would need to get at least 15 rep to be able to flag. I'd rather see experienced users vouch for a tool, than a tool with the power of 5 users vouching for itself.
– CerbrusMar 5 '18 at 10:48

3

@Cerbrus not to mention that the latter is against the Rules As Written.
– John DvorakMar 5 '18 at 10:51

4

@Cerbrus, you have 215 users now casting random flags on random site, I doubt the overview would be different. We are already trusting Charcoal to keep an eye on things, more then we trust 1 of those 215 random users.
– Petter FribergMar 5 '18 at 11:35

I really like the idea of having Smokey accounts to both remove the badge incentive (yes that's a thing for some people) and communicate clearly to mods where the flags are coming from.
– Monica CellioMar 5 '18 at 14:08

2

Brad's comment is easily countered when you consider the fact that moderators have the ability to automatically delete content unilaterally, which is even more powerful than what this entails. The only difference is that moderators get elected (and many moderators have come from such organized chatrooms, so it's not like these are completely untrustworthy users compared to saints/white knights)
– TylerHMar 5 '18 at 15:38

1

If the concern is really that strong re: badges and flag count / stats, then frankly we should probably talk about this behavior getting set to 100% accuracy threshold and rolled into the Community user and monitored by devs and CMs instead of users.
– TylerHMar 5 '18 at 15:40

3

@TylerH We'd love to do that, but it's SE dev time. There have been a couple efforts to do that; they've all fizzled out. It's not worth integration for the benefit over what we have now (especially if we could reliably nuke spam <10s after creation)
– UndoMar 5 '18 at 15:53

I feel that the increased automated flags should be opt-in with the agreement of the moderation team, and community involved. At a broader level, I think the individual communities ought to have a say when any project may affect how it runs.

To me, the right workflow here would be to let the moderators be the point of contact, have the option of discussing and making a decision as representatives of the community and/or bringing it on meta. I don't feel comfortable with letting any organisation (Other than SE) making this decision for us.

And I think this is a big part of the issue here - that it takes things out of the hands of the individual community to a bigger extent than I'm comfortable with.

Answers: (a) Yes. (b). That kinda is the main benefit, as well as fewer humans having to deal with spam. (c) There are stats in the post that answer that one - average TTD for 1 flag is ~190s, 3 flags ~50s, 5 flags ~10s. (d) No data, even anecdotal. I'd hope not, but it might happen on the smaller sites.
– ArtOfCodeMar 5 '18 at 5:19

17

One benefit: I have a huge database of things you really don't want to look at for a variety of reasons. Nasty, nasty stuff that's designed to haunt you for a long time. Getting between 2 and 5 human eyes off of that is a huge benefit IMO.
– UndoMar 5 '18 at 5:21

1

Regarding that example where they voted without knowing...that's exactly what this post and the previous post I made were for. It informed the community about this project and allowed them to sign up to participate. It's not like we randomly selected a user to flag as. They had to authorize the use of their account. They clicked the button to do so. They read the warning
– AndyMar 5 '18 at 5:23

2

Also on the nuking accounts thing - most spammers we see use an account once and then abandon it. It's debatable whether leaving them there does any damage, really, since their profiles aren't linked from any posts (all nuked) and they're not going to use them again.
– ArtOfCodeMar 5 '18 at 5:30

3

Other than that profile spamming is a thing, and awareness of existing spam helps get rid of dormant spam accounts.
– Journeyman Geek♦Mar 5 '18 at 5:56

6

@Cerbrus I'd like the folks who are active on, and/or help run the sites decide what's an appropriate level of input, and I'm uncomfortable with the decision on spam primarily made automatically, with minimal input from folks on the site. I'm not comfortable with the decision on how many flags are done being made for us. That's basically what the answer above says.
– Journeyman Geek♦Mar 5 '18 at 11:16

3

Get a mod to request it - we're either elected by the community, or picked to represent them after all. Shouldn't be too hard for one to message or drop a charcoal admin an email.
– Journeyman Geek♦Mar 5 '18 at 11:29

4

The alternative is literally "We're doing this" and despite their good work, I don't think anyone really should be deciding on behalf of the network how things should work.
– Journeyman Geek♦Mar 5 '18 at 11:30

3

@Cerbrus the point is that this group of users isn't involved in all the sites. And they aren't SE employees, they're just a group of volunteers who've declared a war on spam (which they've been winning because, frankly, they're so good at it). This doesn't change the fact that they aren't members of the relevant communities so this proposal would give this group of "outsiders" a certain degree of "power" to decide how this community does things. That's what seems worrying. So giving the choice to the community instead of the group of outside users seems reasonable.
– terdonMar 5 '18 at 11:46

2

@terdon How would that be significantly different with the power outsiders get from the association bonus? A organization aside, outsiders (in the particular SE community sense) are already given plenty of power on a new site they look into, which seems to be by design
– MagischMar 5 '18 at 11:48

2

@Magisch the difference is that when you come in and create an account as an individual user, you then get involved in the community and become a part of it. If you just create an account, never visit the site, and only cast flags, you never have the chance to understand the local culture. And flagging culture does actually vary across the network. Imagine a group of people who love cleaning. They offer to come into all public building and clean them. For free! Wouldn't it be reasonable to first ask the people working in those buildings though?
– terdonMar 5 '18 at 11:51

1

You still actually read the posts you vote on in hnq. And if you were actually there I'd say you're not auto voting. So ... Very bad comparison @magisch.
– Journeyman Geek♦Mar 5 '18 at 11:58

2

@Magisch no, and the HNQs list is one of the most annoying features of SE for precisely that reason. If I could remove the sites I participate in from HNQ I would do so gladly. But that's besides the the point. What JMG is suggesting here is that whether or not outside help is wanted should be down to the communities to decide. Since this meta post is suggesting that this change can be switched off for sites that don't want it, it is presumably possible to do it on a site-by-site basis. If so, it could also be possible to have it off and only turn it on on a site-by-site basis.
– terdonMar 5 '18 at 11:58

7

@Cerbrus that is quite literally a problem that something like this needs to solve. I keep hearing that "its too hard to get input from all 177 sites" - but we're stakeholders - we run the sites and create the content there. A successful project isn't just about statistics and shiny - its about people, and communication and certain aspects of it feel like autofac . "YOU HAVE SPAM! WE CLEAN SPAM! MORE CLEAN!" when we actually want folks keeping an eye out for spam the old fashioned way too.
– Journeyman Geek♦Mar 5 '18 at 12:09

5

Cause if smoke detector/charcoal goes down, those users would be our backup. There's certain situations where as a mod, I'd actually like to see spam flags so I know what's going on. I occasionally get slightly confused spam flags under "other". Its harder for me to nuke accounts. And well, at some point spammers are going to evolve their tactics anyway, so I would rather a system that works well, and keeps people involved in the loop for "strange things" over just getting rid of all of it with minimal intervention cause "its more efficient.
– Journeyman Geek♦Mar 5 '18 at 12:28

I am grateful to the Charcoal and SmokeDetector developers and users for their substantial, commendable efforts to fight spam on Stack Exchange sites.

However, in relation to this specific proposal, I want to comment on a specific point, which is made twice by the OP:

It will only take 1 or 2 manual flags from users to spam-nuke an autoflagged post, instead of the current 3. [...]

There are currently 3 manual flags required in addition to the automatic flags to nuke a post; this increase proposal still retains at least one manual flag.

(Emphasis mine.)

A reduction to two manual flags would still permit human peer review, i.e. confirmation or disputation of a human's decision by another human. This seems to me to be the minimal essential safety check against humans going rogue. It is less safe than requiring three manual flags, but at least it does not forego peer review altogether; and of course it would cut the humans' workload by (about) a third. I would be supportive of this.

A reduction to one manual flag would eliminate this crucial safety check. I would not be supportive of this.

From one of the graphs above, four flags isn't a significant difference from 3. We have systems in place that review everything reported retroactively. I plan to modify those to require at least two eyes on everything. I don't know of an instance where we've flagged a false positive and missed it - but you're welcome to review the data for yourself.
– UndoMar 5 '18 at 21:10

1

@Undo, "[using] four [machine] flags [out of six] isn't a significant difference from [using three out of six]". That's considering only one metric: time to delete. But on the metric of human workload, the difference presumably would be, as I mentioned in my answer, much more significant: a reduction of (about) a third.
– sampablokuperMar 5 '18 at 22:05

@Undo, "I plan to modify those to require at least two eyes on everything." The implication that you would do this in the future seems to contradict the part of the OP's proposal that says "Multiple people review each post we catch, whether it's autoflagged or not", which suggests the system already is set up such that it will require human peer review even if the proposal is implemented. (As an aside: if the system does already require human peer review, then either the proposal makes no sense, or else it uses "nuke" in a very misleading way. "Nuke" indicates irrevocable deletion.)
– sampablokuperMar 5 '18 at 22:14

3

Currently, you have to have at least three people look at everything before it's deleted. With 5 automatic flags, it would only be one person to nuke. I'm going to add another person to review after the nuke so we can't miss any false positives (which, historically, are 1 in thirty thousand) - essentially keeping at least two eyes watching for false positives.
– UndoMar 5 '18 at 22:16

@Undo, "Review after nuke" is self-contradictory, for the reason given in my previous comment, i.e. that in the context of computer systems, "nuke", like "shred", means irrevocable deletion. Please can you write comprehensibly?
– sampablokuperMar 5 '18 at 22:18

@sampablokuper - "nuke" in this case means deletion. What Undo means is looking at what happened after the fact - reviewing the situation in hindsight, after the post has been deleted.
– MithicalMar 5 '18 at 22:20

2

Nuking a post is reversible by a ♦ moderator. It's something we don't want to happen; it's extremely unlikely to happen, but if it does, we can fix things together.
– GlorfindelMar 5 '18 at 22:21

1

Glorfindel and Mithrandir, perhaps "nuking" means something different in Elrond, but if the OP wants their post to be understood by the average SO user, it would help a lot if the term could be used according to the common convention in computing contexts, i.e. to mean deleting irrevocably. If you & Undo & the OP are appealing to some kind of in-group anti-pattern redefinition of common terms, then that practice should be dropped before attempting to bring others into the discussion, or else entirely avoidable misunderstanding will occur, as has happened here :(
– sampablokuperMar 5 '18 at 22:31

4

I agree with sampablokuper here, @Undo - it'd be kinda nice to nuke the "nuke" terminology, since it suggests something rather more... final than what is actually being discussed here. Trivia: I have a bookmarklet on my phone named "nukethis" that takes whatever post is on my screen, offensive-deletes it, and then destroys its author with a reason that ensures their IP can't post anything for the next little while. That's about as close as we can get to "nuke" here.
– Shog9Mar 6 '18 at 3:45

@sampablokuper for better or for worse, "spam-nuke" is the most common (short) term I've seen for the 6-flag = auto-deletion mechanism. I think I've even seen some of SO's mods call it that :/
– mbrigMar 6 '18 at 17:24

For clarification, when we ‘review’ a post we’re generally looking at a mirrored copy that we store in our web dashboard, that’s why we can still see it after it’s been ‘nuked’ (or deleted)
– angussidneyMar 6 '18 at 20:23

@angussidney, thanks for the clarification. However (and this is not directed at you personally, just at the groupthink that resulted in it happening): storing "a mirrored copy ... in our web dashboard" - i.e. migrating, archiving, or quarantining - is so far from what "nuke" conventionally means in computing that I can't even begin to express how mind-bogglingly misleading it is to use that word to describe it.
– sampablokuperMar 6 '18 at 21:18

Related to the issues of transparency, perhaps an automated comment under a post that's been autoflagged would be helpful in understanding and sighting Smoke Detector activity in situ so to speak, as opposed to needing to make a search of Meta Smoke or a monthly report, as suggested by Shog.

Of course these tools could still be made available, but an auto comment sits with the post and is more easily accessible. It would also show any potential manual flaggers, that it's been auto-flagged. To assist user's not knowing about the project the auto comment could include a link to the Charcoal wiki or a canonical post on Meta, perhaps the FAQ, when/of Smokey is added to the FAQ.

The spammers will figure out we have a more aggressive system for removing spam and will spam more aggressively.
– Sonic the Stay-Home HedgehogMar 6 '18 at 4:50

2

which feeds the spam filters, which gets rid of them faster, Don't forget, smokey feeds spamram. I actually think some feedback - maybe via one of the flags being custom might be a nice touch over the current system
– Journeyman Geek♦Mar 6 '18 at 4:55

5

@Ano In the unlikely event that spammers review their own posts (they don't) it would not be unequivocally a bad thing for them to discover that we dislike them and are taking action. Sure, most are obviously unrepentant, and giving them too much details about our countermeasures would be stupid. But I don't think this is anywhere near crossing that line.
– tripleeeMar 6 '18 at 5:50

@Ano erm, keen spammers can already read all this? And will have access to everything the general community has.
– user310756Mar 6 '18 at 6:21

1

@Ano if anything, it will make the spammers realize they better spend their time and resources on other sites, with less spam protection. :)
– Shadow Keeps Social DistanceMar 6 '18 at 7:22

2

There are several good reasons NOT to have automated comments coming from Smoke Detector or its users. The primary reason is that SD users can't actually do anything to help someone whose post has been automatically deleted - that requires a moderator. "Hey, we deleted your post for [reasons] - if we got it wrong, please help us train our classifier!" isn't an awesome introduction to the site. Beyond that, we know from experience that telling spammers exactly what got their spam identified just results in a long, tedious process of banning every possible unicode representation of a phone #
– Shog9Mar 6 '18 at 17:49

6

But probably the best reason not to do comments is simply that they're comments - the expectation of most readers is that there's a human behind them who is going to be responsive to feedback. When that's not forthcoming, there's no telling what the reaction will be. I've had spammers do public record searches and threaten my family; you don't normally expect that from a spam flag, because it's completely anonymous. Expecting volunteers to stand by their mistakes is fine; expecting them to tolerate abuse isn't.
– Shog9Mar 6 '18 at 17:54

@Shog9 well I was thinking Smokey would make the auto comment - so stalking smokey will be hard - but if you think it's going to make it easier for spammers - the thing is - they can click on the spam tag on any site and they will find all they need to know - maybe the comment will make it easier. Yep as for expecting a reply - the user could be directed to charcoal chat. Anyway- it's no big deal - it was an idea thrown into the ring for evaluation, and it sounds like it's not such a good idea. Sorry about your family being threatened - too many psychos out and about
– user310756Mar 7 '18 at 0:53

Suggestion #1: site transparency

Provide a clear list of what was flagged and removed as spam (either as a separate review queue, or just a list). This way the site community will have an option to gamify getting another Gold badge for reviews review the spam removal history for accuracy.

Suggestion #2: user transparency

Make it crystal clear to a user that their post was removed as spam, with an obvious link of how to dispute that (and a solid workflow procedure to dispute). This may not address 100% of false positives some of which may be from new/unregistered users; but it will at the very least hopefully reduce the amount of grief and upset from false positived users.

I just posted something similar, but disagree vehemently with #2 - this is essentially handing a roadmap to spammers / trolls. As with any mishandled post, a wronged user should flag for moderator attention and let them determine what action is needed - there shouldn't be two paths to redress for users based on who flagged them. What's missing right now is tooling that allows moderators quick access to Smoke Detector from such a post; fortunately, that's much easier to fix.
– Shog9Mar 5 '18 at 22:58

@Shog9 - I should have been more clear but #2 was proposing just that - a flag to moderators. I merely wanted to make it obvious and visible for benefit of some random newbie who never heard of moderator flag and won't know what the little "flag" link does; via a custom message we have on, say, duplicate VTC Qs
– DVKMar 6 '18 at 0:37

We do do a bit better now about showing the reason for deletion on posts, @DVK - but flag is all there is, for anyone. That and /contact. This is what a spam-deleted post looks like to its author - making that more explanatory is outside the scope of this discussion (incorrectly-deleted posts happen rarely, but happen without Smoke Detector's involvement more than with it). That said, don't hesitate to propose a more intuitive UI there if you have ideas.
– Shog9Mar 6 '18 at 3:54

1

@Shog9 - imho, merely adding something like "If you think this was marked as spam in error, click the 'flag' link to alert the moderators" seems more than sufficient.
– DVKMar 6 '18 at 4:00

Note of warning I have not read all of the answers here. I'm sure what I wrote is duplicated somewhere.

I like smokey, it works amazingly well. Yes I would trust it to fully automatically delete spam.

I'm going to point out some "issues" and then speek to them (we are on meta after all).

There are two issues with moving from 3 flags to 5.

Do we really need the extra speed? On the site I moderate, we do not get that much spam. (Smokey has auto flagged 8 posts, total). With the user base, and the amount of questions we get per day we really don't need a machine gun on our spam. A friendly bear watching over the site seems sufficient for us.

Humans are far more likely to get something wrong then a machine. Yes smokey is correct somewhere around 99.7%, but what is the human error rate? (I've made mistakes, clicked the wrong button etc.) Now with only one human vote, you've removed the human self check. A human error could get by with just one person voting. I don't like that idea. (There are users on BSE I would not trust with that responsibility, and I know there are some on your sites too.) We would essentially be giving whoever came across that auto flagged post moderator powers.

I understand that smokey would be auto flagging only the most obvious spam posts. Posts that SE should be filtering out.

In that light I have no problem with smokey moving to 5 flags. I trust smokey, and if it is that good at finding the really bad spam I think there is no way of a user to mess up. Say we have a spam post, if the user thinks it is not spam (somehow) then the post waits in the queue for the next guy.

About the speed, It's spam. The quicker we get rid of it the better. We may not "need" the speed, but there is no harm in having spam disappear quicker.

Go for it.

Remember... Only YOU Can Prevent Spam Posts.

Please note that only posts that could get 5 automated flags, are those of which the system is absolutely certain it's spam. Out of the nearly 30k spam posts in SD's history that meet the strict criteria to receive 5 auto-flags, there hasn't been a single false positive. Any post that doesn't meet those criteria won't get 5 auto-flags, which in turn means more users will need to get involved to get less spammy posts deleted. Basically, the system is conservative, by design.
– CerbrusMar 6 '18 at 20:38

I'm not against this idea - but using actual user accounts to do botted automation is a dodgy hack that should be rectified before pushing this further in my opinion. Fair enough while it was an experiment - but now this is getting a little bit beyond the experiment phase.

How reasonable would it be to create a SmokeDetector user that has the privilege to cast multiple spam votes? I think if we got that happening we would fix the transparency issues highlighted in other answers.

Once this transparency is achieved, I would feel comfortable with the idea of bumping it up to 4 votes (ie, 2 manual votes still required). Once we trial that for a while, running the graphs again should give us insight as to how we should proceed.

I do not like the idea of using fake / separate accounts for this. Philosophically, that breaks tradition: it allows whatever user controls the account(s) extra privilege over other users. Practically-speaking, it removes the cost of getting it wrong: everyone signing up for this does so with the understanding that they're still responsible for their actions, even if they allow those actions to be dictated to some extent by SD; they have their accounts to lose if they take that responsibility lightly.
– Shog9Mar 6 '18 at 4:06

3

@Shog9 - That's actually a very convincing point of view... A good thing about a special user is that if it goes haywire there is only one user to ban (and presumably it would make it relatively easy to find all the damage it caused). I also imagined that such a user would have little other privileges (ie, it can't vote/answer/comment/etc) so it wouldn't be a useful account excepting for spam flagging. But the potential loss of accountability is a very good counterargument...
– ShadowMar 6 '18 at 4:24

1

@Shadow If the thing goes haywire, we have other ways of dealing with that. It'll get shut down very quickly by whoever notices, and if necessary we can revoke the API access tokens we have.
– ArtOfCodeMar 6 '18 at 5:36

2

@Shog9 and Shadow, I continue to disagree; it would be cleaner with dedicated bots account, in the end the bot developers are always responsible (a random selection of 300 users is very hard to keep responsible), it's messy for moderators and it's gaming flags and badge (not that I care, but some do). Maybe yeah if instead the developers account were used but probably 1 bot account connected to a 1 single developers would be best, but I guess it is best we discuss this on another meta and concentrate on number on flags that SD should cast on this meta.
– Petter FribergMar 6 '18 at 20:37

1

Ok, so you do want dedicated accounts, @Petter? I think that's something we'd have to be very, very careful with; the entire system is designed around individual responsibility, and a bot account separates that from any one individual. Try to focus on what problems such an arrangement would solve, and how we might reduce the risk of error or abuse.
– Shog9Mar 6 '18 at 20:43

2

@Shog9 Yes either developer accounts (like Andy, comment bot) or 1 single bot account connected to developer profile (like BR, natty), the developer can't flag when bot does, this makes it clear for moderator what is going on (they know why flagged), the flag goes either to developer (merit) or to bot account, but this rule is not directly related to SD more a general rule. This random picking a user from a +300 users base seems messy and without merit.
– Petter FribergMar 6 '18 at 20:48

3

To flag 5 times you just would need 5 developers (hence probably not a problem for SD), but yeah the users connected to bot account can't flag themself If the bot does a messy you just follow the developer link in profile and you will find out who is to blame and can fix stuff
– Petter FribergMar 6 '18 at 20:49

2

Just for sake of argue, immagine you have 300 users and miss flag 300 times, that however would be only 1 miss flag per user (how would you blame that user?). This is not the case for SD but just trying to explain how the logic of responsibility fails, also since badges are a thing for some, what merit did these 300 users have?, how can a moderator easily identify that it was auto-flagged if they see random users (they can't even get experience on who is who)
– Petter FribergMar 6 '18 at 21:09

1

I think you should write an answer, @Petter; there's a lot to unpack in those last two comments. While you're writing that, consider this: one bot would be insufficient. Quite possibly 5 bots would be insufficient. A user with 15 rep starts out with 10 flags and that doesn't increase significantly until there are hundreds of past helpful flags - so on a site with a spam problem you'd need a lot of bots. If you wanted to raise 5 flags/post on 40 posts a day, that'd be 200 bots minimum. Nevermind the crazy optics of even creating 200 network-wide sockpuppets.
– Shog9Mar 6 '18 at 22:29

1

Naw, @FTP is right, I'm not thinking clearly here. 20 bots is still a problem, but less of one than 200.
– Shog9Mar 6 '18 at 23:25

1

@Shog9 Fundamentally, allowing a bot to act on my behalf separates responsibility (mine) from control (bot's). If its actions cause me to be banned, I have no recourse, except to deactivate it, and apologize. I can't see how this is better than having dedicated accounts, where responsibility is placed directly on the developer, rather than implicitly so.
– jpaughMar 7 '18 at 15:29

1

That is recourse, @jpaugh. You're using a tool - that doesn't absolve you of responsibility. In practice, if something went horribly wrong we'd probably disable both the tool and the folks using it until the problem could be sorted out. But as with Petter's concerns, this probably should be a separate discussion if we're discussing hypothetical situations.
– Shog9Mar 7 '18 at 16:03

@Shog9 Yes, that is status quo; which is why I won't enable smoke detector, no matter how accurate it is.
– jpaughMar 7 '18 at 16:40

1

I'm with @jpaugh on this. And if the SE interface asserts that user X has taken an action, other users should be able to trust this assertion. Currently, they can't. Something like en.wikipedia.org/wiki/Wikipedia:Bot_policy would be far more transparent & accountable. 3rd-party bots shouldn't masquerade as human users for the sake of any CRUD operation except maybe retrieval. I'm disappointed SE considers it acceptable, let alone on this scale. Spam-fighting is important, but hardly justifies such a serious abuse of accountability as this.
– sampablokuperMar 8 '18 at 0:03

My concern is alerting users in general to this. Perhaps there is advice to avoid getting one's question/post inadvertently marked as spam. This may be especially important to any links posted ... Such as 'do' this or 'don't do' that, etc....

There's no review queue built into Stack Exchange for spam, no. The only review queue that spam realistically gets put into is the one that we operate on metasmoke, Charcoal's web dashboard.
– ArtOfCodeMar 7 '18 at 3:37

1

Perhaps that's worth a change. One spam flag, auto or manual, and reviewers are alerted. Spam is bad, but not bad enough to risk losing good posts due to four erroneous auto-flags and one mistake or accident by a human.
– WGroleauMar 7 '18 at 4:26

1

That'd be worth a feature-request here on MSE. It's not something Charcoal can implement since we're not on the SE team, but I'd be happy to see it.
– ArtOfCodeMar 7 '18 at 4:37

Rather than needing accounts with special privileges such as "multiple flags per post" make, and register system-wide, 5 accounts. CharcoalAutoflag1 through 5, say, or come up with something better. They would perhaps need extra flagging volume, but nothing special flag-per-post. They simply pile on in the same way that real people do. Indeed, if they are accurate, they could even get their extra flagging volume the same way real users do, by having their flags classified as helpful.

If it's:

99% spam, CharcoalAutoflag1 flags it.

99.5 CharcoalAutoflag2 also flags it

99.75 CharcoalAutoflag3 also flags it

99.95 CharcoalAutoflag4 also flags it

99.99 CharcoalAutoflag5 also flags it (or 99.999 or however many decimal places you want to put there.)

I agree with the general idea that the automated flagging seems good from a spam-removal view, but if it's pretending to be real users doing the flagging, it's not at all apparent if a post is being flagged by people or bots. Having whichever account correlates to the highest level of certainty post a comment to the effect that "Charcoal's automated spam detection evaluates this post as 99.5% likely to be spam" or similar message would also be helpful (perhaps you are doing this, I've hardly spent the evening combing though the whole backstory to formulate the commentary on a small part of the process I had not known about until today.)

If you edit spam out of a spam post, it is much less likely to get flagged as spam. Editing spam out is counter-productive in getting the spammer banned.
– CerbrusMar 7 '18 at 14:40

SmodeDetector isn't leaving any automated comments. Making these automated flags recognizable as such is indeed an concern, but I'm not sure comments are the solution.
– CerbrusMar 7 '18 at 14:42

@Catija I have edited to remove the incidental comment portion that has you all ignoring the content related to the question. You may delete the 3 now-irrelevant comments and get back to the actual question.
– EcnerwalMar 7 '18 at 16:47

You can feel free to blame that part of your post for the downvotes but you're not the first person here suggesting using special accounts, so you might actually read the other answers before picking on that as an explanation. As an example, see the comments on this answer. Your answer may be more detailed, but it still suggests something that's unlikely to be a good choice.
– Catija♦Mar 7 '18 at 16:55

You've missed the reason why multiple accounts are required - each account has limited flags. Although your idea makes sense up until that point, the "99%" would need to be distributed between each bot otherwise the 1st bot will run out of flags way before the 5th one.
– ShadowMar 8 '18 at 0:22

One of the reasons why this doesn't work is that spam happens in waves. We would need a whole bunch of accounts with the assoc bonus, because a small site like say graphic design could get almost nothing for a year and suddenly 20 posts a day out of left field.
– MagischMar 8 '18 at 7:39