Definition
Core Concept
Central Limit Theorem (CLT): states that distribution of sample means of independent, identically distributed (i.i.d.) variables approaches a normal distribution as sample size increases.
Scope
Applies to sums, averages, and other linear combinations of random variables with finite mean and variance.
Significance
Foundation for inferential statistics: justifies normal approximations in hypothesis testing, confidence intervals.
Historical Background
Origins
First versions by Abraham de Moivre (1733): approximated binomial distribution by normal curve.
Development
Laplace extended CLT for sums of i.i.d. variables (1810). Lyapunov and Lindeberg refined conditions in 20th century.
Modern Formulation
Theorem formalized with rigorous assumptions on moments and independence by Lyapunov (1901), Lindeberg (1922).
Formal Statement
Setup
Let {X₁, X₂, ..., Xₙ} be i.i.d. random variables, mean μ, variance σ² < ∞.
Theorem
As n → ∞, distribution of standardized sum:
Zₙ = (ΣXᵢ - nμ) / (σ√n)Interpretation
Sampling distribution of sample mean approaches normality regardless of population shape.
Conditions and Assumptions
Independence
Random variables must be independent or weakly dependent under certain extensions.
Identically Distributed
Classic CLT requires identical distribution; generalized versions relax this.
Finite Mean and Variance
Existence of finite expected value μ and variance σ² crucial for convergence.
Sample Size
Larger samples yield better normal approximation; no fixed n suffices for all distributions.
Intuition and Interpretation
Aggregation of Randomness
Sum of many small, random effects tends to smooth distribution towards bell curve.
Robustness
Insensitivity to original distribution shape; skewness and kurtosis lessen with n.
Practical Implication
Enables use of normal models in diverse fields: economics, biology, engineering.
Mathematical Proofs
Characteristic Functions
Proof via convergence of characteristic functions to that of normal distribution.
Lyapunov’s Condition
Uses Lyapunov’s moment condition for non-identical variables to ensure convergence.
Lindeberg’s Condition
Generalizes CLT to triangular arrays with Lindeberg’s condition controlling variance contributions.
Applications
Statistical Inference
Confidence intervals, hypothesis testing rely on approximate normality of sample means.
Quality Control
Control charts assume normality in process averages for defect detection.
Finance
Modeling returns and risk assessment under central limit assumptions.
Machine Learning
Bootstrap methods and ensemble models use CLT for error estimation.
Examples
Binomial to Normal
Binomial distribution B(n,p) approximated by normal when np and n(1-p) large.
Sample Mean of Uniform Variables
Average of uniform(0,1) samples approaches normal as n increases.
Dice Rolls
Sum of fair dice rolls converges to normal shape with increasing rolls.
Limitations and Extensions
Non-Finite Variance
CLT fails for variables with infinite variance (e.g., Cauchy distribution).
Dependent Variables
Extensions handle weak dependence, mixing sequences, Markov chains with modified conditions.
Rate of Convergence
Berry-Esseen theorem quantifies speed of convergence to normality.
Multivariate CLT
Generalizes to vector-valued variables converging to multivariate normal.
Tables and Formulas
Summary of CLT Conditions
| Condition | Requirement |
|---|---|
| Independence | Variables must be independent or weakly dependent |
| Identical Distribution | Classic CLT requires same distribution |
| Finite Mean | Mean μ exists and finite |
| Finite Variance | Variance σ² exists and finite |
| Sample Size | Large enough for approximation (varies by distribution) |
Key Formulas
Zₙ = (ΣXᵢ - nμ) / (σ√n)lim (n→∞) P(Zₙ ≤ z) = Φ(z)where:Xᵢ = i.i.d. variables,μ = E[Xᵢ],σ² = Var(Xᵢ),Φ(z) = standard normal cdf. Berry-Esseen Bound
| Bound | Description |
|---|---|
| |Fₙ(x) - Φ(x)| ≤ Cρ/σ³√n | Uniform bound on cdf difference; ρ = third absolute moment. |
References
- Feller, W., An Introduction to Probability Theory and Its Applications, Vol. 2, Wiley, 1971, pp. 544–547.
- Billingsley, P., Probability and Measure, 3rd ed., Wiley, 1995, pp. 338–342.
- Durrett, R., Probability: Theory and Examples, 4th ed., Cambridge University Press, 2010, pp. 244–250.
- Lyapunov, A. M., General Problem of the Stability of Motion, Annals of Mathematics, 1901, 2(4), 105–140.
- Lindeberg, J. W., Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung, Mathematische Zeitschrift, 1922, 15(1), 211–225.