Probability Mass Function

Definition

Discrete Random Variable

Discrete random variable: variable taking countable values. Each value associates with a probability. Probability mass function (pmf): function assigning probabilities to each discrete outcome.

Formal Definition

PMF denoted as p_X(x) or P(X = x). It satisfies: p_X(x) = P(X = x), for all x in the sample space.

Domain and Range

Domain: set of all possible discrete values. Range: probabilities in [0, 1], sum equals 1.

p_X: R → [0,1]such that ∑_{x ∈ Support} p_X(x) = 1and p_X(x) ≥ 0 ∀ x

Properties

Non-negativity

All probabilities are non-negative: p_X(x) ≥ 0 for every outcome.

Normalization

Total probability sums to one: ∑ p_X(x) = 1 over all x.

Support

Support: set of x where p_X(x) > 0. Outside support, probability is zero.

Discrete Nature

Defined only on countable values. Not applicable for continuous variables.

Relation to Cumulative Distribution Function (CDF)

Definition of CDF

CDF, F_X(x), is probability that X ≤ x. For discrete variables: F_X(x) = P(X ≤ x).

Connection between PMF and CDF

CDF is cumulative sum of PMF values up to x: F_X(x) = ∑_{t ≤ x} p_X(t).

Recovering PMF from CDF

PMF is difference of successive CDF values: p_X(x) = F_X(x) - F_X(x^-), where x^- denotes previous point.

Examples of PMF

Bernoulli Distribution

Outcomes: 0 or 1. Parameter: success probability p. PMF:

p_X(x) = { p if x=1 1-p if x=0 0 otherwise }

Binomial Distribution

Number of successes in n trials. PMF:

p_X(k) = C(n,k) p^k (1-p)^{n-k} for k=0,1,...,n

Poisson Distribution

Counts events in fixed interval. Parameter λ. PMF:

p_X(k) = (e^{-λ} λ^k) / k! for k=0,1,2,...

Calculation and Formulation

From Frequency Data

PMF estimated as relative frequency: p_X(x) ≈ (count of x) / (total observations).

Analytical Derivation

PMF derived from known probabilistic models or assumptions.

Law of Total Probability

PMF can be expressed via conditioning:

p_X(x) = ∑_y P(X=x | Y=y) P(Y=y)

Expectation and Variance Using PMF

Expectation

Expected value E[X] computed as weighted sum:

E[X] = ∑_x x p_X(x)

Variance

Variance Var(X) measures spread:

Var(X) = E[(X - E[X])^2] = ∑_x (x - E[X])^2 p_X(x)

Higher Moments

Higher order moments: E[X^n] = ∑_x x^n p_X(x).

Joint Probability Mass Function

Definition

Joint PMF of discrete variables X and Y: p_{X,Y}(x,y) = P(X=x, Y=y).

Properties

Non-negativity, normalization: ∑_x ∑_y p_{X,Y}(x,y) = 1.

Marginal PMF

Marginals obtained by summing out other variable:

p_X(x) = ∑_y p_{X,Y}(x,y)p_Y(y) = ∑_x p_{X,Y}(x,y)

PMF vs PDF

Discrete vs Continuous

PMF applies to discrete variables. PDF applies to continuous variables.

Probability Calculation

PMF: probability of exact value. PDF: probability density; probability over interval via integral.

Mathematical Differences

PMF sums to one. PDF integrates to one.

Applications in Probability and Statistics

Modeling Discrete Phenomena

Applications: coin tosses, dice rolls, event counts, quality control.

Statistical Inference

Used in parameter estimation, hypothesis testing for discrete data.

Machine Learning

Discrete probabilistic models (e.g., Naive Bayes) rely on pmf.

Limitations and Constraints

Inapplicability to Continuous Variables

PMF undefined for continuous variables; requires PDF instead.

Computational Complexity

Large support sets increase computational load for exact pmf calculations.

Data Requirements

Estimating pmf accurately requires sufficient data for frequency estimates.

Computational Aspects

Storage

PMF stored as arrays, dictionaries for finite or countable supports.

Numerical Stability

Care needed when probabilities are very small to avoid underflow.

Software Tools

Statistical packages (R, Python's scipy.stats) provide pmf functions.

Summary

Probability mass function: core concept for discrete random variables. Defines probability distribution over countable outcomes. Enables calculation of expectation, variance, and supports joint distributions. Key tool in theoretical and applied probability.

References

• Feller, W. Introduction to Probability Theory and Its Applications, Vol. 1, Wiley, 1968, pp. 50-110.
• Ross, S. M. A First Course in Probability, 9th ed., Pearson, 2014, pp. 75-130.
• Casella, G., Berger, R. L. Statistical Inference, 2nd ed., Duxbury, 2002, pp. 25-70.
• Grimmett, G., Stirzaker, D. Probability and Random Processes, 3rd ed., Oxford University Press, 2001, pp. 45-90.
• Billingsley, P. Probability and Measure, 3rd ed., Wiley, 1995, pp. 60-120.