Can sentiment analysis spot sarcasm?

In my work helping clients manage the risks of social media, twitter hashtag-hijacking often comes up in conversation. A spectacular example is UKIP's recent use of the hashtag #WhyImVotingUKIP as a way of letting their supporters explain the reasons behind their vote.

In my work helping clients manage the risks of social media, twitter hashtag-hijacking often comes up in conversation. A spectacular example is UKIP's recent use of the hashtag #WhyImVotingUKIP as a way of letting their supporters explain the reasons behind their vote.

Unfortunately for UKIP, the hashtag was hijacked and was used by many as a way of poking fun at UKIP. If you read the many tweets you'll notice that they are filled with a lot of sarcasm. Some seem to be clearly sarcastic jokes, others are a little more difficult to work out if it's a joke or serious. This got me thinking. If I'm struggling to understand the sentiment in some of the tweets then analytical sentiment software has no chance.

It is widely accepted that one of the most challenging elements of sentiment analysis is identifying sarcasm. I've been thinking about how we may overcome this obstacle and, having recently attended a PwC data analytics event, I've been thinking what else we can pull into the equation to get to a more accurate sentiment score.

While many sentiment analysis apps focus on analysing individual tweets, perhaps doing some deeper analysis of a user's tweets may give us a more accurate prediction of the sentiment used and help us identify sarcasm. You could also dig deeper and overlay other data, such as voting data for a particular county or borough with the geo-locational data that may have been included in the tweet. Combining these data sets could allow you to predict more accurately whether a tweet contains sarcasm or not.

Of course, while what I describe is analysis of publicly available data it's likely that data privacy questions would be raised. Companies should be cautious when considering the extent to which they will mine twitter data so as not to lose trust.