Posted
by
Soulskill
on Friday July 05, 2013 @04:12PM
from the meta-comment-challenge:-complete-sincerity dept.

Nerval's Lobster writes "Now here's the greatest thing ever: French tech firm Spotter has apparently devised an analytics platform capable of identifying sarcastic comments, according to the BBC. Spotter's platform scans social media and other sources to create reputation reports for clients such as the EU Commission and Air France. As with most analytics packages that determine popular sentiment, the software parses semantics, heuristics and linguistics. However, automated data-analytics systems often have a difficult time with some of the more nuanced elements of human speech, such as sarcasm and irony — an issue that Spotter has apparently overcome to some degree, although company executives admit that their solution isn't perfect. (Duh.) Spotter isn't alone: IBM, Salesforce, and other IT vendors are hard at work on analytics software that can more perfectly determine when you're mouthing off, you little punks. In theory, sarcasm detection can help with customer service, and judging how well products are doing on the open market... and we all know it's going to work perfectly, right? Nothing could possibly go wrong with automated platforms built to assess the nuances of human speech."

But this only deals with one specific instance, if you have a poster with a history of tinfoil hat posts say "Sure, I totally believe NSA has only my best interests at heart" then that has a lot higher probability of being irony than a poster that is fully in the "if you have nothing to hide, you have nothing to fear" corner. Given all the defective sarcasm and irony detectors out there, the bar of out-detecting a human is pretty damn low.

The problem comes with professional violators of Poe's Law, such as Stephen Colbert's character, "Steven Colbert of the Colbert Report". He's a parody of every right wing nut job talk show host. His schtick is to take a right-wing agenda item and push it beyond its obvious short term benefits to its logical but socially detrimental conclusion, where he continues to defend it even more vigorously using Republican platform talking points, ad hominem attacks, and every other logical fallacy [wikipedia.org] he can throw at it. He does this consistently without ever breaking character. And he has a flock of brilliant writers who are able to help him pull this off night after night.

As a matter of fact, he is so consistent that he was mistaken for an actual right wing comedian, and was invited to speak at the White House Correspondent's Dinner in 2006 where he lampooned George W. Bush to his face for fifteen straight minutes. Very few of the faithful present laughed at the routine. President Bush turned red almost from the get-go, politely grimaced out a smile, sat through the entire speech, and left the stage immediately after Colbert finished. I have no doubt that heads rolled within five minutes. ( My favorite joke from the event went something like, " 'Those naysayers claim that this administration is just rearranging the deck chairs on the Titanic.' That is a terrible metaphor. This administration is not sinking. This administration is soaring! If anything, they are rearranging the deck chairs on the Hindenburg!" )

No one, and I mean no one, who is paying attention and able to parse the audience interaction could possibly mistake Colbert for the right wing nut job that he is parodying.

Well, a detailed academic study of hundreds of people who were shown parts of the Colbert Report demonstrated that you're wrong about this [sagepub.com]. From the abstract:

conservatives were more likely to report that Colbert only pretends to be joking and genuinely meant what he said while liberals were more likely to report that Colbert used satire and was not serious when offering political statements.

I personally find it hard to believe that no one involved in inviting him to a White House event would have realized the extent of Colbert's sarcasm. But clearly many "normal" conservatives don't get it....

Sarcasm in general does not fare well in text media. As practiced in spoken language, there's a very significant component of intonation and body language. Of course it can be done and well in text media, but it's a skill most people don't have, like writing good expository prose.

Can you translate into and out of 50 different languages? Google Translate and the Systran engine can.

Let me fix that for you: Can Humans translate into and out of 50 different languages?Why yes, yes we can, and we do a far better job of than Google.

But we can't program a computer to translate a language we don't know. And if we can't distinguish between a parody of extremism, or subtle sarcasm reliably as humans without visual or written clues, how would you propose to tell a machine to do so?

If they make it work and ever point that at slashdot, the readings are gonna be flying off the charts!

Slashdot would be a poor test bed for the project. Sarcasm is too easily detected on here to be useful, it's as subtle as being hit by a brick.

Now... if they pointed it at Faux Nooz, that would be pretty interesting to see how much the presenters don't believe of the garbage they're spewing to keep the market other broadcasters have neglected: the disenfranchised intelligentsia.

Sarcasm is very frequently indicated by nuances that aren't transmitted through text. If humans have trouble getting sarcasm out of text, why should an algorithm do any better with the same set of data?

Perhaps we should have a new computer keyboard that senses biological changes in the typist,
to infer things like sarcasm; typing rate, skin conductivity, pulse, body temp, combine with webcam,
add facial expressions.

The lack of a sarcasm mark is a serious one. Why is sarcasm a second-rate method of expressing oneself through text when compared to declarations, imperative statements, interrogations, shouting or unfinished sentences?

Their great technology will make it work! It is so much better than what puny humans can do and not bound by the limitations of text. In fact, I predict the core technology is an advanced quantum-bogon-detector, that will even be able to classify statements before they are made or if they are not made at all!

Sarcasm is very frequently indicated by nuances that aren't transmitted through text. If humans have trouble getting sarcasm out of text, why should an algorithm do any better with the same set of data?

Lisa: Now next week is our "state of the city" address. Has everyone finished their proposals?Comic Book Guy: Well first of all I've a plan to eliminate obesity in women.Lyndsey Nagle: Oh please, for a nickel-a-person tax increase we could build a theatre for shadow puppets.Dr. Hibbert: Balinese or Thai?Lyndsey Nagle: Why not both, then everybody's happy.Comic Book Guy: Oh yeah, everyone's real happy then.Lyndsey Nagle: Do I detect a note of sarcasm?Professor Frink: (With sarcasm detector) Are you kidding

I can't believe that I'm reading a comment that is so clearly uninformed like yours on Slashdot.

Every language has its tradeoffs. It's apparent that you've not only disregarded the benefits of OO languages, but that you've also failed to correctly identify their drawbacks. Put two equally competent developers up against each other, one in C and one in C#, and I'd put my money on the C# app being more reliable and stable out of the gate, and quite possibly more efficient as well, since optimization takes tim

You just replied to a single parenthesized comments in someone's post with a 5 paragraph paper that is too long to read.

I just want to point it's relatively non-controversial that very simple C or Assembly programs are much more robust under load than large and complicated programs written in an Object-oriented language.

This is a function of complexity and feature bloat, and OO languages impose a very high amount of complexity and feature bloat, that adversly impacts small simple applications.

I believe I agree with everything you said, though I will point out that comparing a "simple" application against a "large and complicated" one in terms of performance is not exactly a fair comparison. Even so, I think I get what you meant, and I certainly agree that rewriting critical paths in a lower-level language can have significant benefits. I wasn't attempting to argue otherwise. I was, however, attempting to argue that the AC had made a sweeping generalization that was unwarranted.

Often the detection of sarcasm relies on understanding of popular opinion on a topic. I don't think we'll have any magic bullet algorithm to detect sarcasm until we have hard AI with a far-reaching corpus of current knowledge.
Take these two sentences: "DRM is the best. It makes everything so much easier!" and "The iPhone is the best! It makes everything so much easier!"
Ok, algorithm. Pick the one containing sarcasm...

Is not like you won't end in jail for a sarcastic comment [cnn.com], or get expelled over a joke [theblaze.com], it will work in the other way, seeing sarcams where they aren't and getting you anyway. And getting this mess in your private mail, where you usually joke and don't care a lot about potential readings of what you say, because, well, you don't have anything to hide, will make life interesting in the next years.

And brought to justice! How dare they making legal and ethical NSA interception and interpretation of all communication harder! That amounts to terrorism! Time to find all these thought-criminals and lock them away for good. All clear speaking and thinking citizens will live in a better world for that.

In general it's not a good idea to mod such things down, it's better to reply to them. Because if one person wooshed, other people will too; which means that someone else with mod points will come by later to 'fix' your downmod.

Modding is primarily to get rid of spam and GNAA, and secondarily to bring attention to really interesting posts. It's not there to act as an adjudicator between 'right' and 'wrong.' That's why there's no -1 Wrong mod option.

I agree absolutely with the second line, and agree tentatively with the first line.

I would further suggest that limited mod points are a serious factor limiting the growth of Slashdot. People don't want to comment if they have no chance of being modded up. More mod points would help that.

EU Commission could save some money here. The algorithm to detect sarcasm when speaking of EU Commission is simple, as nobody ever tells anything good about the EU Commission : If a sentence has a positive word, then it is sarcasm.

Everyone is missing the point. The real question is why do they want a sarcasm detector?

Implicitly we all realize that they want to filter sarcastic remarks out of online posting. Sarcasm is a very effective way to combine criticism and humor, and the result can be a very effective critique. This makes it very troublesome to those with power and money. They don't want anyone rocking the boat or getting uppity.

So instead of addressing potentially meaningful critical responses, or accepting the reality that people enjoy making bad jokes, they seek to automate the process of self serving censorship.

The intent is bad. I'm sure that organizations considering using this technology don't care about false positives. What they want is for you to STFU, unless you say what they want you to say.

So while Slashdot posters make the truly obvious jokes, or argue about technology and false positive/negative rates, this reveals the ugly truth about the intent of big online organizations. They want to enforce a one way channel where users are censored. Considering that Slashdot considers itself to be an elite corner of the internet, I find it pathetic that no one has a clue about what this means.

They are just trying to measure public opinion by sorting comments into "positive" and "negative". They can already sort out most of the negative comments, their shortfalling is false positives due to sarcsm. If their purpose was to censor negative opinion, then they would already be censoring all those non-sarcastic posts they don't like.

In other words, I have no clue how you got modded up because your argument makes no sense.

And can easily be achieved without any sarcasm detection.Social media mentions have a strong positive bias, so simply guessing positive all the time will get youpretty close to the 80% mark. and with a simple list of negative phrases you can pass the 80% mark in identifying if a social media mention(twitter, facebook, etc.) is positive or negative.

This whole idea that sarcasm doesn't come through in text needs to be revisited.

I have a reputation in my work environment for being perceptive, thoughtful, and lucid. I also have a reputation for having near perfect recall of anything previously discussed that could possibly go wrong, and for sometimes becoming extremely intense and hard to deter from constantly injecting these unhappy reminiscences into self-satisfied negotiations until everyone else glasses over. Others might characterize this as a gee