Definition
Basic Concept
Probability Density Function (PDF): function describing relative likelihood of continuous random variable taking a value near x. Denoted as f(x). Integral over interval gives probability.
Mathematical Definition
For continuous random variable X, PDF f(x) satisfies:
f(x) ≥ 0 for all x ∈ ℝ∫_{-∞}^{∞} f(x) dx = 1Interpretation
f(x) not probability itself but density. Probability over interval [a, b] is area under curve:
P(a ≤ X ≤ b) = ∫_{a}^{b} f(x) dx"Probability densities embody the continuous analog to discrete probabilities, enabling precise quantification of uncertainty across intervals." -- William Feller
Properties
Non-negativity
f(x) ≥ 0 for all x. Ensures valid probability interpretation.
Normalization
Total area under f(x) equals 1:
∫_{-∞}^{∞} f(x) dx = 1Probability Calculation
Probability over any interval equals integral of f(x) over that interval.
Support
Set of x where f(x) > 0. Outside support, PDF equals zero.
Continuity
PDF often continuous, but can have discontinuities if distribution is mixed.
| Property | Description |
|---|---|
| Non-negativity | f(x) ≥ 0 ∀x |
| Normalization | ∫ f(x) dx = 1 |
| Support | Set of x with f(x) > 0 |
Relation to Cumulative Distribution Function
Definition of CDF
Cumulative Distribution Function (CDF) F(x) = P(X ≤ x). Non-decreasing, right-continuous, limits 0 and 1.
Connection Between PDF and CDF
PDF is derivative of CDF wherever derivative exists:
f(x) = dF(x)/dxRecovering CDF from PDF
CDF found by integrating PDF:
F(x) = ∫_{-∞}^x f(t) dtComputing Probabilities
Probability Over Interval
Calculate probability X lies between a and b:
P(a ≤ X ≤ b) = ∫_{a}^{b} f(x) dxPoint Probability
For continuous variables, P(X = x) = 0 because integral over zero-width interval.
Using Improper Integrals
Probabilities over infinite intervals use improper integrals:
P(X > c) = ∫_{c}^{∞} f(x) dxNumerical Integration
When analytic integration impossible, apply numerical methods: trapezoidal, Simpson’s rule, Monte Carlo.
Examples of PDFs
Uniform Distribution
Constant PDF over finite interval [a, b]:
f(x) = 1/(b - a), a ≤ x ≤ b; 0 otherwiseExponential Distribution
Models waiting times. Parameter λ > 0:
f(x) = λ e^{-λ x}, x ≥ 0; 0 otherwiseNormal Distribution
Bell-shaped curve, parameters μ (mean), σ (std dev):
f(x) = (1/(σ√(2π))) e^{-(x - μ)^2 / (2σ^2)}Beta Distribution
Defined on [0,1], parameters α, β > 0:
f(x) = (x^{α-1} (1-x)^{β-1}) / B(α, β), 0 ≤ x ≤ 1Common Continuous Distributions
Normal Distribution
Symmetric, unimodal, central limit theorem foundation. Parameters μ, σ.
Exponential Distribution
Memoryless property, models time until event.
Gamma Distribution
Generalizes exponential. Parameters shape k, rate θ.
Beta Distribution
Flexible on [0,1], models proportions.
Chi-Squared Distribution
Sum of squares of k independent standard normal variables. Used in hypothesis testing.
| Distribution | Parameters | Support |
|---|---|---|
| Normal | μ, σ > 0 | (-∞, ∞) |
| Exponential | λ > 0 | [0, ∞) |
| Beta | α, β > 0 | [0, 1] |
Applications
Statistical Modeling
PDFs model real-world continuous phenomena: heights, weights, times.
Signal Processing
Noise distributions modeled by PDFs (e.g., Gaussian noise).
Reliability Engineering
Failure time distributions inform maintenance schedules.
Machine Learning
Density estimation, anomaly detection, probabilistic classifiers.
Physics and Engineering
Quantum mechanics, thermodynamics describe system states probabilistically.
Estimation and Inference
Parameter Estimation
PDF form determined by parameters estimated via Maximum Likelihood, Method of Moments.
Kernel Density Estimation
Non-parametric method to approximate PDF from data samples.
Hypothesis Testing
Test statistics distributions modeled by PDFs for p-value computation.
Confidence Intervals
PDFs used to derive intervals containing true parameter with specified probability.
Multivariate Probability Density Functions
Definition
Joint PDF f(x₁, x₂, ..., xₙ) describes likelihood of vector-valued continuous random variables.
Properties
Non-negative, integrates to one over ℝⁿ. Marginal PDFs obtained by integrating out variables.
Conditional PDFs
Define conditional density via joint and marginal PDFs:
f(x|y) = f(x,y) / f(y), f(y) > 0Example: Bivariate Normal
Joint PDF with mean vector μ and covariance matrix Σ:
f(x) = (1 / (2π |Σ|^{1/2})) exp(-1/2 (x - μ)^T Σ^{-1} (x - μ))PDF vs PMF
Definition
PDF: continuous variables, PMF: discrete variables.
Value Interpretation
PDF values are densities, not probabilities. PMF values are probabilities.
Probability Calculation
PDF: probability via integration; PMF: probability via summation.
Example Comparison
Discrete variable X ∈ {1, 2, 3} has PMF values; continuous variable Y on ℝ has PDF.
Limitations and Considerations
Interpretation Challenges
PDF values alone lack probabilistic meaning without integration context.
Non-existence for Some Variables
Not all distributions have PDFs (e.g., singular distributions).
Computational Complexity
Integral calculations may be difficult, require approximations.
Mixed Distributions
Some variables combine discrete and continuous parts, requiring hybrid approaches.
Summary
Probability Density Function (PDF) defines distribution of continuous random variables. Key properties: non-negativity, normalization, relation to CDF. Enables calculation of probabilities via integration. Central in statistics, engineering, science. Understanding PDFs critical for modeling uncertainty and analyzing data.
References
- Billingsley, P. Probability and Measure, Wiley, 3rd ed., 1995, pp. 100-150.
- Casella, G., Berger, R.L. Statistical Inference, Duxbury, 2nd ed., 2002, pp. 250-300.
- Feller, W. An Introduction to Probability Theory and Its Applications, Vol. 2, Wiley, 1971, pp. 200-260.
- Ross, S.M. Introduction to Probability Models, Academic Press, 11th ed., 2014, pp. 120-180.
- Papoulis, A., Pillai, S.U. Probability, Random Variables and Stochastic Processes, McGraw-Hill, 4th ed., 2002, pp. 90-130.