Liu, Bingwei

A typical method to obtain valuable information is to extract the sentiment or opinion
from a message. Machine learning technologies are widely used in sentiment classification
because of their ability to "learn" from the training dataset to predict or support
decision making with relatively high accuracy. However, when the dataset is large,
some algorithms might not scale up well. In this paper, we aim to evaluate the scalability
of Naïve Bayes classifier (NBC) in large datasets. Instead of using a standard library
(e.g., Mahout), we implemented NBC to achieve fine-grain control of the analysis procedure.
A Big Data analyzing system is also design for this study. The result is encouraging
in that the accuracy of NBC is improved and approaches 82% when the dataset size increases.
We have demonstrated that NBC is able to scale up to analyze the sentiment of millions
movie reviews with increasing throughout.