Definition and Basic Concepts
Random Variable Types
Random variable: function mapping outcomes to real numbers. Discrete: countable values. Continuous: uncountably infinite values over intervals.
Continuous Random Variable
Definition: variable with range as an interval or union of intervals on real line. Realizations: any value within range possible.
Motivation
Model phenomena with infinite precision: time, height, temperature. Captures uncertainty in measurements and natural processes.
Probability Distributions
Concept
Mapping values to probabilities. For continuous variables, probabilities for exact values are zero; probabilities over intervals are positive.
Distribution Function
Describes likelihood across domain. Must satisfy monotonicity and limit properties.
Support
Set of values where probability density > 0. Defines effective domain of the random variable.
Probability Density Function (PDF)
Definition
Function f(x) such that P(a ≤ X ≤ b) = ∫_a^b f(x) dx. Must satisfy f(x) ≥ 0 and ∫_−∞^∞ f(x) dx = 1.
Interpretation
Density: relative likelihood of variable near x. Not a probability itself but integral over interval is probability.
Properties
Non-negativity, normalization, integral over support equals 1.
| Property | Description |
|---|---|
| Non-negativity | f(x) ≥ 0 for all x |
| Normalization | ∫_−∞^∞ f(x) dx = 1 |
Cumulative Distribution Function (CDF)
Definition
F(x) = P(X ≤ x) = ∫_−∞^x f(t) dt. Non-decreasing, right-continuous, limits: 0 at −∞, 1 at ∞.
Relationship to PDF
Derivative: f(x) = dF(x)/dx almost everywhere. CDF integrates PDF.
Uses
Calculating probabilities for intervals, quantiles, defining medians and percentiles.
Expectation and Moments
Expected Value
Mean or first moment: E[X] = ∫_−∞^∞ x f(x) dx. Measure of central tendency.
Higher Moments
n-th moment: E[X^n] = ∫_−∞^∞ x^n f(x) dx. Describe shape, skewness, kurtosis.
Moment Generating Function (MGF)
M_X(t) = E[e^{tX}]. Generates moments via differentiation. Exists for some distributions only.
Expectation:E[X] = ∫_{−∞}^{∞} x f(x) dxn-th Moment:E[X^n] = ∫_{−∞}^{∞} x^n f(x) dxMoment Generating Function:M_X(t) = E[e^{tX}] = ∫_{−∞}^{∞} e^{tx} f(x) dxVariance and Standard Deviation
Variance
Measure of spread: Var(X) = E[(X − μ)^2] = E[X^2] − (E[X])^2.
Standard Deviation
Square root of variance: σ = √Var(X). Same units as variable.
Properties
Non-negative, additive for independent variables, scale and shift transformations.
| Formula | Description |
|---|---|
| Var(X) = E[X²] − (E[X])² | Variance definition |
| σ = √Var(X) | Standard deviation |
Common Continuous Distributions
Overview
Families of distributions modeling various phenomena. Key examples: normal, uniform, exponential, beta, gamma.
Classification
Symmetric vs asymmetric, bounded vs unbounded, light-tailed vs heavy-tailed.
Selection Criteria
Data characteristics, domain constraints, shape, tail behavior.
Normal Distribution
Definition
Continuous distribution with bell-shaped curve. Parameters: μ (mean), σ² (variance).
PDF Formula
f(x) = (1 / (σ√(2π))) * exp(−(x−μ)² / (2σ²))Properties
Symmetric, unimodal, mean=median=mode, infinitely differentiable. Central Limit Theorem foundation.
Uniform Distribution
Definition
Equal probability over interval [a,b]. Parameters: a (lower bound), b (upper bound).
PDF Formula
f(x) = 1 / (b−a), for x ∈ [a,b]; 0 otherwiseProperties
Constant density, mean = (a+b)/2, variance = (b−a)² / 12. Model of maximum uncertainty within bounds.
Exponential Distribution
Definition
Models waiting times between Poisson events. Parameter: λ > 0 (rate).
PDF Formula
f(x) = λ e^{−λx}, for x ≥ 0; 0 otherwiseProperties
Memoryless property: P(X > s+t | X > s) = P(X > t). Mean = 1/λ, variance = 1/λ².
Applications
Engineering
Signal processing, reliability analysis, noise modeling, system lifetimes.
Finance
Modeling asset returns, risk assessment, option pricing (Black-Scholes assumes normality).
Natural Sciences
Measurements in physics, biology (growth rates), environmental modeling (rainfall, temperature).
Simulation and Sampling
Random Number Generation
Pseudo-random generators produce uniform(0,1). Transformations yield other continuous distributions.
Inverse Transform Method
Generate U~Uniform(0,1). Compute X = F^{-1}(U) where F is CDF of target distribution.
Rejection Sampling
Sample from proposal distribution, accept/reject based on ratio. Used when inverse CDF unknown.
Inverse Transform Algorithm:1. Generate U ~ Uniform(0,1)2. Compute X = F^{-1}(U)3. Return X as sample from desired distributionReferences
- Casella, G., Berger, R. L. Statistical Inference. Duxbury, 2002, pp. 120-180.
- Ross, S. M. Introduction to Probability Models. Academic Press, 2014, vol. 11, pp. 50-95.
- Feller, W. An Introduction to Probability Theory and Its Applications. Wiley, 1968, vol. 1, pp. 200-250.
- Billingsley, P. Probability and Measure. Wiley, 1995, 3rd ed., pp. 150-210.
- Gut, A. Probability: A Graduate Course. Springer, 2013, pp. 95-140.