Definition and Basic Properties
Moment Generating Function (MGF) Definition
For a random variable X, the moment generating function M_X(t) is defined as the expectation of e^{tX}:
M_X(t) = E[e^{tX}], \quad t \in \mathbb{R}Domain: values of t for which this expectation is finite.
Interpretation
MGF encodes moments of X via derivatives at zero. Acts as a Laplace transform of the distribution.
Basic Properties
- M_X(0) = 1
- M_X(t) ≥ 0 if X is non-negative
- MGF is convex in t on its domain
- MGF uniquely determines the distribution if it exists in a neighborhood of zero
Existence and Domain
Conditions for Existence
MGF exists if E[e^{tX}] < ∞ for t in an open interval around zero. Not all distributions have MGFs (e.g., Cauchy).
Domain of MGF
Set of t ∈ ℝ where M_X(t) converges. Can be finite or infinite interval.
Relation to Distribution Tails
Exponential tails imply larger domain; heavy tails often restrict domain.
Relation to Moments
Moment Extraction
n-th moment given by n-th derivative of MGF at zero:
E[X^n] = M_X^{(n)}(0) = \frac{d^n}{dt^n} M_X(t)\bigg|_{t=0}Generating All Moments
MGF generates all moments if it exists in an open neighborhood around zero.
Example: Mean and Variance
Mean: E[X] = M'_X(0). Variance: Var(X) = M''_X(0) - (M'_X(0))^2.
Examples of MGFs
Discrete Distributions
- Bernoulli(p): M_X(t) = 1 - p + p e^{t}
- Poisson(λ): M_X(t) = exp(λ(e^{t} - 1))
Continuous Distributions
- Normal(μ, σ²): M_X(t) = exp(μt + (σ² t²)/2)
- Exponential(λ): M_X(t) = λ / (λ - t), t < λ
Table of Common MGFs
| Distribution | MGF M_X(t) | Domain |
|---|---|---|
| Bernoulli(p) | 1 - p + p e^t | All real t |
| Poisson(λ) | exp(λ(e^t - 1)) | All real t |
| Normal(μ, σ²) | exp(μ t + σ² t² / 2) | All real t |
| Exponential(λ) | λ / (λ - t) | t < λ |
Uniqueness Theorem
Statement
If two random variables have identical MGFs in an open interval around zero, then their distributions are identical.
Implications
MGF characterizes distribution uniquely if it exists locally. Used in distribution identification.
Proof Sketch
MGF is Laplace transform of distribution measure. Injectivity of Laplace transform implies uniqueness.
Cumulant Generating Functions
Definition
Cumulant generating function (CGF) K_X(t) = log M_X(t).
Relation to Cumulants
n-th cumulant κ_n = K_X^{(n)}(0). Represents joint moments subtracting dependencies.
Advantages
CGF simplifies moment calculations, especially for sums of independent variables (additivity).
Operations on MGFs
Sum of Independent Random Variables
If X, Y independent, M_{X+Y}(t) = M_X(t) · M_Y(t).
Scaling
For aX, M_{aX}(t) = M_X(at).
Mixtures
MGF of mixture is weighted sum of component MGFs.
MGFs and Convergence in Distribution
Continuity Theorem
Pointwise convergence of MGFs in a neighborhood of zero implies convergence in distribution.
Central Limit Theorem
Proofs often use MGFs to show normalized sums converge to normal distribution.
Limitations
MGF may not exist for all distributions; characteristic functions sometimes preferred.
Applications in Probability and Statistics
Moment Calculation
Derive moments and cumulants efficiently.
Distribution Identification
Recognize distribution by analytic form of MGF.
Statistical Inference
Used in likelihood methods, hypothesis testing, and generating functions of estimators.
Limitations and Alternatives
Non-Existence Cases
MGF fails to exist for heavy-tailed or undefined exponential moments (e.g., Cauchy).
Characteristic Functions
Always exist, defined as E[e^{i t X}]. Preferred in general theory.
Other Generating Functions
Probability generating functions (PGFs) for discrete nonnegative r.v.s; Laplace transforms for positive variables.
Computational Aspects
Numerical Evaluation
MGF often computed via numerical integration or closed-form expressions where available.
Symbolic Differentiation
Used to obtain moments symbolically from MGF derivatives.
Software Tools
Mathematica, R, Python libraries (SymPy, SciPy) support MGF computations and moment extraction.
Summary and Key Points
- MGF M_X(t) = E[e^{tX}] encodes all moments if it exists in a neighborhood of zero.
- Uniquely determines distribution under existence conditions.
- Cumulant generating function simplifies moment calculations.
- MGFs facilitate analysis of sums, scaling, and convergence.
- Limitations include non-existence for some distributions; characteristic functions provide alternatives.
References
- Feller, W. An Introduction to Probability Theory and Its Applications, Vol. II, Wiley, 1971, pp. 290-320.
- Billingsley, P. Probability and Measure, 3rd ed., Wiley, 1995, pp. 155-175.
- Gut, A. Probability: A Graduate Course, Springer, 2005, pp. 60-85.
- Durrett, R. Probability: Theory and Examples, 4th ed., Cambridge University Press, 2010, pp. 85-98.
- Casella, G., Berger, R. L. Statistical Inference, 2nd ed., Duxbury, 2002, pp. 190-210.