Improving Usual Naive Bayes Classifier Performances with Neural Naïve Bayes based Models

Naive Bayes is a popular probabilistic model appreciated for its simplicity and interpretability. However, the usual form of the related classifier suffers from two significant problems. First, as caring about the observations’ law, it cannot consider complex features. Moreover, it considers the conditional independence of the observations given the hidden variable. This paper introduces the original Neural Naive Bayes, modeling the classifier’s parameters induced from the Naive Bayes with neural network functions. This method allows for correcting the first default. We also introduce new Neural Pooled Markov Chain models, alleviating the conditional independence assumption. We empirically study the benefits of these models for Sentiment Analysis, dividing the error rate of the usual classifier by 4:5 on the IMDB dataset with the FastText embedding, and achieving an equivalent F1 as RoBERTa on TweetEval emotion dataset, while being more than a thousand times faster for inference.