Toggle navigation sidebar

Toggle in-page Table of Contents

Probability & Statistics

Introduction

Chapter 1. Mathematical Preliminaries

Permutations and Combinations
Calculus
Contour Maps
Exercises

Chapter 2. Probability

Probability Space
Probability Axioms
Conditional Probability
Independence
Baye’s Theorem and the Law of Total Probability
Summary

Chapter 3. Discrete Random Variables

Random Variables
Discrete Random Variables
Probability Mass Function
Cumulative Distribution Function
Expectation
Moments and Variance
Discrete Uniform Distribution
- Implementation
Bernoulli Distribution
- Implementation
Independent and Identically Distributed (IID)
Binomial Distribution
- Implementation
- Real World Examples
Geometric Distribution
Poisson Distribution
- Implementation
Important
Exercises

Chapter 4. Continuous Random Variables

From Discrete to Continuous
Continuous Random Variables
Probability Density Function
Expectation
Moments and Variance
Cumulative Distribution Function
Mean, Median and Mode
Continuous Uniform Distribution
Exponential Distribution
Gaussian Distribution
Skewness and Kurtosis
Convolution and Sum of Random Variables
Functions of Random Variables

Chapter 5. Joint Distributions

From Single Variable to Joint Distributions
Joint PMF and PDF
- Concept
Joint Expectation and Correlation
- Concept
Conditional PMF and PDF
- Concept
- Exercises
Conditional Expectation and Variance
- Concept
- Exercises
Sum of Random Variables
- Concept
Random Vectors
- Concept
Multivariate Gaussian Distribution

Chapter 6. Sample Statistics

Moment Generating and Characteristic Functions
Probability Inequalities
- Probability Inequalities
- Application: Learning Theory
Law of Large Numbers

Notations

Mathematical Notations
Machine Learning Notations
Deep Learning Notations

Machine Learning

The Machine Learning Framework
Fundamentals
Linear Models
Generative
- Naive Bayes
Model Selection and Evaluation
- Metrics and Scoring Rules
  - Classification Metrics
  - Regression Metrics
Trees, Forests, Bagging and Boosting
- Decision Trees
  - Braindump
- Ensemble Learning
Dimensionality Reduction
- Principal Component Analysis
  - Concept
  - PCA
  - Eigenface
Clustering
- K-Means
Neighbourhood
- K-Nearest Neighbours

Deep Learning

Deep Learning
Natural Language Processing (NLP)
- Vector Semantics and Embeddings

Optimization

Gradient Descent
- Gradient Descent Concept
- Gradient Descent Construction

References and Resources

Bibliography
Resources

Powered by Jupyter Book

Binder
Colab

repository
open issue

.ipynb

Implementation

Implementation#

This section implements the linear regression algorithm from scratch.

previous

Concept

next

Logistic Regression

By Gao Hongnan
© Copyright 2022.