Naive Bayes

This section talks about Naive Bayes Classifier.

Further Readings

Books and Lectures

• Zhang, Aston and Lipton, Zachary C. and Li, Mu and Smola, Alexander J. “Chapter 22.7 Maximum Likelihood.” In Dive into Deep Learning, 2021.

• Chan, Stanley H. “Chapter 8.1. Maximum-Likelihood Estimation.” In Introduction to Probability for Data Science, 172-180. Ann Arbor, Michigan: Michigan Publishing Services, 2021

• Zhang, Aston and Lipton, Zachary C. and Li, Mu and Smola, Alexander J. “Chapter 22.9 Naive Bayes.” In Dive into Deep Learning, 2021.

• Hal Daumé III. “Chapter 9.3. Naive Bayes Models.” In A Course in Machine Learning, January 2017.

• Murphy, Kevin P. “Chapter 9.3. Naive Bayes Models.” In Probabilistic Machine Learning: An Introduction. Cambridge (Massachusetts): The MIT Press, 2022.

• James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. “Chapter 4.4.4. Naive Bayes” In An Introduction to Statistical Learning: With Applications in R. Boston: Springer, 2022.

• Mitchell, Tom Michael. Machine Learning. New York: McGraw-Hill, 1997. (His new chapter on Generate and Discriminative Classifiers: Naive Bayes and Logistic Regression)

• Jurafsky, Dan, and James H. Martin. “Chapter 4. Naive Bayes and Sentiment Classification” In Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Noida: Pearson, 2022.

• Bishop, Christopher M. “Chapter 4.2. Probabilistic Generative Models.” In Pattern Recognition and Machine Learning. New York: Springer-Verlag, 2016

Notebooks

• Goodboychan: Naive Bayes and Logistic Regression

• Nathaniel Dake: Bayes Classifiers

Online Resources

• The Bayes Rules Book.

• Wikipedia: Naive Bayes Classifier

• Wikipedia: Joint Probability Distribution

• Maximum Likelihood Estimation of Gaussian Parameters

• mathematicalmonk’s (ML 4.3) MLE for univariate Gaussian mean

• Machine Learning from Scratch

• Cornell CS4780/5780: Machine Learning, Bayes Classifier and Naive Bayes

• On the importance of the i.i.d. assumption in statistical learning

• Time and space complexity of Naive Bayes

• What is the number of parameters needed for a joint probability distribution?

Code Implementations

• Scikit-learn

• https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/supervised_learning/naive_bayes.py

• https://github.com/rushter/MLAlgorithms/blob/master/mla/naive_bayes.py

• https://github.com/oniani/ai/blob/main/models/gaussian_naive_bayes.py

• https://github.com/piyush01123/Revisiting-Classical-ML/blob/master/src/classification/nv_bayes/naive_bayes.py