Normal Distribution

Definition

Concept

Normal distribution: continuous probability distribution. Shape: symmetric, unimodal, bell-shaped curve. Description: variable values cluster around mean. Range: entire real line (-∞, ∞).

Historical Context

Origin: Abraham de Moivre (1733), later formalized by Carl Friedrich Gauss. Applications: error analysis, natural and social sciences.

Mathematical Expression

Defined by probability density function (PDF) with parameters mean (μ) and variance (σ²).

Properties

Symmetry

Distribution symmetric about mean μ. Skewness = 0.

Kurtosis

Excess kurtosis = 0; mesokurtic distribution.

Moments

Mean = μ; variance = σ²; all odd central moments (except 1st) zero.

Moment Generating Function

M(t) = exp(μt + ½σ²t²).

Closure Properties

Sum of independent normal variables: normal. Linear transformations: normal.

Parameters

Mean (μ)

Location parameter. Center of distribution. Expected value E(X).

Variance (σ²)

Scale parameter. Measures dispersion. Variance = E[(X - μ)²].

Standard Deviation (σ)

Square root of variance. Units same as variable.

Parameter Roles

μ shifts curve horizontally. σ controls spread and peak height.

Probability Density Function

Formula

f(x) = (1 / (σ √(2π))) * exp(- (x - μ)² / (2σ²))

Interpretation

f(x): likelihood density at point x. Integral over range = probability.

Graphical Features

Peak at μ. Inflection points at μ ± σ. Area under curve = 1.

Table of PDF Values (μ=0, σ=1)

Cumulative Distribution Function

Definition

CDF F(x) = P(X ≤ x). Integral of PDF from -∞ to x.

Formula

F(x) = (1/2)[1 + erf((x - μ) / (σ √2))]

Error Function (erf)

Special function related to Gaussian integrals. No closed-form in elementary functions.

Properties

Monotonic increasing. Limits: F(-∞)=0, F(∞)=1.

Standard Normal Distribution

Definition

Special case: μ=0, σ=1. Denoted Z ~ N(0,1).

Standardization

Transform any normal variable X by Z = (X - μ)/σ to standard normal.

Table Usage

Standard normal tables provide probabilities for Z-values. Widely used in hypothesis testing.

Symmetry

Probability properties: P(Z ≤ -z) = 1 - P(Z ≤ z).

Central Limit Theorem

Statement

Sum/average of large number of i.i.d. random variables converges to normal distribution regardless of original variable distribution.

Conditions

Independence, identical distribution, finite variance.

Implications

Justifies normality assumption in many statistical methods.

Rate of Convergence

Depends on skewness and kurtosis of original distribution.

Applications

Statistics

Modeling errors, hypothesis testing, confidence intervals.

Natural Sciences

Measurement errors, biological traits distribution.

Engineering

Signal processing, quality control.

Finance

Modeling asset returns, risk management.

Social Sciences

IQ scores, standardized test results.

Parameter Estimation

Method of Moments

Estimators: sample mean (μ̂), sample variance (σ̂²).

Maximum Likelihood Estimation (MLE)

Parameters maximize likelihood function of observed data.

Sample Formulas

μ̂ = (1/n) Σ xᵢσ̂² = (1/n) Σ (xᵢ - μ̂)²

Properties

MLE unbiased for μ, biased for σ²; bias corrected by dividing by n-1.

Normality Tests

Purpose

Assess if data follows normal distribution.

Common Tests

Shapiro-Wilk, Kolmogorov-Smirnov, Anderson-Darling, Jarque-Bera.

Test Statistics

Compare empirical data distribution to theoretical normal.

Interpretation

High p-value: fail to reject normality; low p-value: reject normality.

Limitations

Assumption of Symmetry

Fails for skewed data distributions.

Outlier Sensitivity

Heavily influenced by extreme values.

Heavy Tails

Underestimates probability of extreme events in some contexts.

Applicability

Not suitable for discrete or bounded variables.

Extensions and Generalizations

Multivariate Normal Distribution

Generalizes normal to vector variables with covariance matrix.

Truncated Normal Distribution

Normal distribution bounded by limits.

Skew Normal Distribution

Introduces skewness parameter to model asymmetry.

Mixtures of Normals

Combination of multiple normal distributions to model complex data.

References

• Feller, W. Introduction to Probability Theory and Its Applications, Vol. 2, Wiley, 1971, pp. 145-162.
• Casella, G., Berger, R. L. Statistical Inference, Duxbury Press, 2002, pp. 200-220.
• Rice, J. A. Mathematical Statistics and Data Analysis, Duxbury Press, 2006, pp. 100-115.
• Lehmann, E. L., Romano, J. P. Testing Statistical Hypotheses, Springer, 2005, pp. 300-321.
• Mendenhall, W., Sincich, T. A Second Course in Statistics: Regression Analysis, Pearson, 2012, pp. 75-90.