We consider the over-fitting problem for multinomial probabilistic Latent Semantic Analysis (pLSA) in collaborative filtering, using a regularization approach. For big data applications, the computational complexity is at a premium and we, therefore, consider a maximum a posteriori approach based on conjugate priors that ensure that complexity of each step remains the same as compared to the un-regularized method. In the numerical section, we show that the proposed regularization method and training scheme yields an improvement on commonly used data sets, as compared to previously proposed heuristics.