Probability & Distributions

Probability Basics

Probability quantifies uncertainty. These rules come from set theory — unions and intersections. Counting techniques like combinatorics (permutations and combinations) are essential for computing probabilities in finite sample spaces.

Conditional Probability & Bayes' Theorem

Bayes' Theorem is the backbone of Bayesian inference and machine learning. It lets us update beliefs with new evidence. Total probability connects this to partition: P(B) = Σ P(B|Aᵢ)·P(Aᵢ).

Discrete Distributions

• Bernoulli: Single trial, p = success. E[X] = p, Var = p(1−p)
• Binomial(n, p): P(X = k) = C(n,k)·pᵏ(1−p)ⁿ⁻ᵏ — uses polynomial expansion
• Poisson(λ): P(X = k) = e⁻λ·λᵏ/k! — models rare events per interval
• Geometric(p): P(X = k) = (1−p)ᵏ⁻¹·p — trials until first success. Connects to exponential functions

Continuous Distributions

For continuous distributions, probabilities are areas under the curve — you need integration. The normal (Gaussian) distribution is the most important, governing everything from measurement error to stock prices.

• Uniform(a, b): f(x) = 1/(b−a) — constant density
• Exponential(λ): f(x) = λe⁻λˣ — time between events
• t-distribution: Used in hypothesis testing with small samples
• Chi-squared: Sum of squared standard normals — used in goodness-of-fit tests

Central Limit Theorem