Module 2 — Probability & Statistics

Probability and Statistics are fundamental to Machine Learning and Transformer models. They help describe uncertainty, data distributions, model predictions, and optimization during training.

Topics

Probability
Joint Probability
Conditional Probability
Bayes' Theorem
Random Variables
Probability Distribution
Expectation
Variance
Standard Deviation
Covariance
Correlation

1. Probability

Probability measures the likelihood of an event occurring.

Formula

P(A)=\frac{\text{Number of Favorable Outcomes}} {\text{Total Number of Outcomes}}

2. Joint Probability

Probability that two events occur together.

Formula

P(A\cap B)

For independent events,

P(A\cap B)=P(A)P(B)

3. Conditional Probability

Probability of event A occurring given that B has already occurred.

Formula

P(A|B)= \frac{P(A\cap B)} {P(B)}

::contentReference[oaicite:0]{index=0}

4. Bayes' Theorem

Updates the probability of an event based on new evidence.

Formula

P(A|B)= \frac{P(B|A)P(A)} {P(B)}

::contentReference[oaicite:1]{index=1}

5. Random Variables

A random variable maps outcomes to numerical values.

Formula

Discrete

X:\Omega\rightarrow\mathbb{R}

Continuous

X\sim f(x)

6. Probability Distribution

A probability distribution describes the probabilities of all possible values.

Discrete Distribution

P(X=x)

Continuous Distribution

f(x)

Properties

\sum_x P(X=x)=1

\int_{-\infty}^{\infty}f(x)\,dx=1

7. Expectation (Expected Value)

The average value of a random variable over many trials.

Formula

Discrete

E[X]= \sum_x xP(X=x)

Continuous

E[X]= \int_{-\infty}^{\infty}xf(x)\,dx

::contentReference[oaicite:2]{index=2}

8. Variance

Variance measures how spread out data is from its mean.

Formula

Var(X) = E[(X-\mu)^2]

Equivalent Formula

Var(X) = E[X^2]-E[X]^2

::contentReference[oaicite:3]{index=3}

9. Standard Deviation

Standard deviation is the square root of variance.

Formula

\sigma = \sqrt{Var(X)}

Population Formula

\sigma= \sqrt{ \frac1N \sum_{i=1}^{N} (x_i-\mu)^2 }

::contentReference[oaicite:4]{index=4}

10. Covariance

Covariance measures how two variables change together.

Formula

Cov(X,Y) = E[(X-\mu_X)(Y-\mu_Y)]

Sample Covariance

Cov(X,Y)= \frac1{n-1} \sum_{i=1}^{n} (x_i-\bar{x}) (y_i-\bar{y})

11. Correlation

Correlation measures the strength of a linear relationship between two variables.

Pearson Correlation

\rho(X,Y) = \frac{Cov(X,Y)} {\sigma_X\sigma_Y}

Range

-1\le\rho\le1

Summary

Concept	Formula
Probability	$P(A)=\dfrac{\text{Favorable}}{\text{Total}}$
Joint Probability	$P(A\cap B)$