Introduction

Welcome to this repository, which is designed to parallel and enhance the concepts presented in the book, Introduction to Probability for Data Science.

Notations

• Mathematical Notations
• Machine Learning Notations
• Deep Learning Notations

Optimization

• Gradient Descent

Machine Learning

• The Machine Learning Framework
• Fundamentals
• Linear Models
• Model Selection and Evaluation
• Trees, Forests, Bagging and Boosting
• Dimensionality Reduction
• Neighbourhood
  • K-Nearest Neighbours
    • Concept
    • Summary
    • K-Nearest Neighbours Decision Boundary
    • Curse of Dimensionality
    • The Importance of Feature Scaling in KNN

Deep Learning

• Numerical Stability and Initialization
• Computer Vision
• Natural Language Processing Specialization
• Natural Language Processing (NLP)

References, Resources and Roadmap

• Bibliography
• Resources

Machine Learning

Machine learning is a dynamic field that sits at the intersection of statistics and computer science. It is an essential aspect of artificial intelligence, allowing systems to learn and improve from experience autonomously. Through its powerful algorithms, machine learning enables machines to identify patterns, make predictions, and derive actionable insights from vast datasets.

Deep Learning

Deep Learning, a subset of machine learning, capitalizes on the power of artificial neural networks to solve intricate problems. This method is remarkably effective at digesting large and complex datasets, and it has proven invaluable across a spectrum of applications, from image recognition and natural language processing to speech recognition and robotics.

The essence of deep learning rests on the foundation of neural networks. However, with the advent of advanced computational power, the scope and impact of these networks have expanded exponentially.

We stand at the precipice of a new epoch in artificial intelligence, powered by deep learning. So fasten your seat belts as we embark on this exciting exploration into the profound realms of machine learning and deep learning!

Probability Theory

Probability Theory is a mathematical framework for computing the probability of complex events. Underlying all of machine learning, probability theory gives us the tools to make reasonable decisions in the face of uncertainty. It allows us to deal with randomness in a principled manner and make rigorous inferences from data. Studying probability theory will provide the foundation to understand random variables, distributions, and the all-important Bayes’ theorem, which plays a pivotal role in many machine learning algorithms.

Optimization

Optimization is the mathematical discipline focused on finding the best solution to a problem given a set of constraints. Many machine learning and deep learning algorithms rely on optimization techniques to minimize the error of prediction or to learn the underlying structure of data. Fundamental optimization methods such as gradient descent and its variants, such as stochastic gradient descent and minibatch gradient descent, are essential to understanding how these learning algorithms work and how to improve their performance.

Statistics

Statistics is the discipline that allows us to collect, analyze, interpret, present, and organize data. It provides the methodology to make inferences and predictions based on data, and to make decisions in the face of uncertainty. In the context of machine learning and deep learning, statistics aids in understanding the data we work with, and in the design and evaluation of our models. Central concepts in statistics, such as the bias-variance tradeoff, overfitting, hypothesis testing, and confidence intervals, are crucial to developing effective machine learning models.

Linear Algebra

Linear Algebra is the branch of mathematics dealing with vector spaces, linear transformations, and matrices. It underpins a significant part of both machine learning and deep learning, as many of the methods used in these fields can be understood and implemented in terms of vectors and matrices. Understanding linear algebra is essential for understanding how data is represented and manipulated within machine learning algorithms, how different layers in a neural network interact, and how complex models can be efficiently computed.

Calculus

Calculus, specifically differential and integral calculus, is vital to machine learning and deep learning. Many machine learning algorithms involve optimization, which requires understanding of gradients and how they are computed—concepts from calculus. Concepts like derivatives and partial derivatives, integrals, limits, and series expansions are critical for understanding how machine learning models learn from data and how this learning process can be improved. In the realm of deep learning, backpropagation, a fundamental algorithm for training neural networks, relies heavily on concepts from calculus.