The Blog

Presidential State of the Union Sentiment Analysis

Sentiment analysis is a hot technology in marketing. This analysis parses comments and feedback left on corporate and social media sites and attempts to discern the tone or attitude expressed by the commenter. For example, reviews and comments about a company’s product such as “I love these shoes” represent a strong positive statement about how the customer feels about that product.

The challenge is that the comments are just free-form text in English which computers traditionally had a hard time deciphering.

M&S Big Data thought it would be interesting to apply this sentiment analysis to our Presidents. As a start we ran the State of the Union through our text mining application. We collected all 225 speeches from 1790 through today, including President Obama’s 2015 address.

Soon we will add enhancements which will deepen the analysis and allow anyone to personally interact with the app so that they can explore the Presidential statements on their own.

We will soon be releasing an interactive version that will be available to the public.

Sentiment Analysis

M&S applied a sentiment score to every word in the State of the Union speeches and calculated the overall “tone” or sentiment of each speech in the sense of being optimistic (positive) or pessimistic (negative).

Below are the results for the period from the year 2000 to the present. You can see that the sentiment calculation matches what common sense would expect. In January, 2000, the tone was very positive. This was President Clinton’s last year in office, the internet boom was in full bloom and the events of 9/11 were yet to happen.

The speech in January 2002 was the first after 9/11 and clearly the sentiment had fallen dramatically. In January 2003 the sentiment was actually negative which is a rare occurrence (the U.S. invasion of Iraq was just a few weeks away).

The tone improved after that but then dropped dramatically again in the January 2009 speech. Why do you think that happened?

Ego Measures

This was just for fun. The graph represents how many times the President talked about himself in each speech! Do you see a relationship between self-references and political party?

Terms

Terms is a more in-depth digging into the actual content of the speeches. First we show you all references to a word (that is, every sentence in which the word occurred). Here are all the references to the word “terrorist”:

For even more technical insight, here are the list of words which are highly correlated with the word “terrorist”:

Word Clouds

And lastly – for the moment – you can experiment with word clouds. Here is a word cloud of the terms used during President Obama’s term in office (to-date):

Technology

All of the analytics were done by M&S Consulting using the R language. We created the interactive visualizations using “Shiny” from RStudio. You can get more information about RStudio at http://www.rstudio.com/. And feel free to reach out to us if you have any questions about R, RStudio, Shiny, and our services. We’re happy to help!

[message_box title=”Big Data and Business Intelligence” color=”blue”]If you or your organization are in need of help or want to learn more, please contact us, we would be happy to talk to you.
Contact Us[/message_box]