Standard Deviation

Definition

Concept Overview

Standard deviation (SD) quantifies data dispersion relative to its mean (expected value). It measures average deviation magnitude, expressed in the same units as the data.

Context in Probability

In probability theory, SD describes spread of a random variable's probability distribution around its expectation.

Purpose

Purpose: assess variability, risk, uncertainty in stochastic processes and datasets.

Notation

Common notation: σ (population SD), s (sample SD).

"Standard deviation provides a natural measure of spread that is intuitive and mathematically tractable." -- Ronald Fisher

Mathematical Formulation

Population Standard Deviation

Definition: square root of the variance of a random variable X with expectation μ.

σ = sqrt(E[(X - μ)²])

Sample Standard Deviation

Estimate from sample {x₁, x₂, ..., x_n}, mean x̄:

s = sqrt( (1/(n-1)) Σ (x_i - x̄)² )

Expectation Operator

E[·] denotes expected value: weighted average of outcomes according to their probabilities.

Discrete and Continuous Cases

Discrete: sum over all outcomes. Continuous: integral over probability density function (pdf).

σ = sqrt( Σ (x_i - μ)² P(X=x_i) ) (discrete)σ = sqrt( ∫ (x - μ)² f(x) dx ) (continuous)

Properties

Non-negativity

σ ≥ 0 always. Zero iff all values identical (no dispersion).

Units

Same units as original data: facilitates interpretation vs variance.

Scale Invariance

Multiplying X by constant c scales SD by |c|: σ_cX = |c| σ_X.

Additivity for Independent Variables

For independent X and Y: Var(X+Y) = Var(X) + Var(Y); SD not additive, but combined variance sums.

Relation to Moments

SD is square root of second central moment: E[(X - μ)²].

Interpretation

Measure of Spread

SD quantifies typical deviation from mean: larger SD implies greater spread.

Probabilistic Meaning

In normal distributions, ~68% values lie within ±1σ of mean; ~95% within ±2σ.

Risk Assessment

Used in finance: SD as volatility metric indicating uncertainty of returns.

Comparison Tool

Compare variability across datasets or experiments with different means.

Calculation Methods

Direct Formula

Calculate mean, then average squared deviations, then square root.

Computational Formula

More efficient formula:

σ = sqrt( E[X²] - (E[X])² )

Sample Calculation Steps

1. Compute sample mean x̄.
2. Calculate squared deviations (x_i - x̄)².
3. Sum and divide by n-1 (Bessel's correction).
4. Take square root.

Software Implementation

Most statistical software provides built-in SD functions; verify population vs sample choice.

Examples

Discrete Data Set

Data: {2, 4, 4, 4, 5, 5, 7, 9}

Mean: 5, Variance: 4, SD: 2

Continuous Distribution

Normal distribution N(μ=0, σ=1): SD equals 1 by definition.

Comparison Between Samples

Sample A: {10, 12, 23, 23, 16, 23, 21, 16}, Sample B: {10, 10, 10, 10, 10, 10, 10, 10}

Sample A SD higher due to variability; B SD = 0.

Applications

Statistics and Data Analysis

Summarizes data variability; essential in descriptive statistics.

Quality Control

Monitors process stability by measuring variation in output.

Finance

Measures asset return volatility; aids portfolio risk management.

Natural Sciences

Quantifies experimental error and measurement precision.

Machine Learning

Feature scaling, normalization, and anomaly detection rely on SD.

Relation to Variance

Variance Definition

Variance (Var(X)) is expectation of squared deviations: E[(X - μ)²].

Standard Deviation as Square Root

SD equals sqrt of variance: σ = √Var(X).

Interpretation Difference

Variance in squared units; SD in original units. SD preferred for interpretability.

Mathematical Implications

Variance additive for independent variables; SD not additive.

Formula Summary

Variance: σ² = E[(X - μ)²]Standard Deviation: σ = sqrt(σ²)

Standard Deviation in Distributions

Normal Distribution

SD defines spread; controls bell curve width.

Bernoulli Distribution

SD = sqrt(p(1-p)), where p is success probability.

Poisson Distribution

SD = sqrt(λ), λ is mean number of events.

Uniform Distribution

SD = (b-a)/√12 for interval [a,b].

Exponential Distribution

SD equals mean: 1/λ.

Limitations

Sensitivity to Outliers

Large deviations disproportionately increase SD; not robust.

Assumption of Symmetry

SD less informative for skewed or multimodal distributions.

Not Always Intuitive

Interpretation assumes data near mean; misleading if distribution heavily tailed.

Sample Bias

Sample SD underestimates population SD without Bessel correction.

Alternative Measures of Dispersion

Mean Absolute Deviation (MAD)

Average absolute deviations from mean; less sensitive to outliers.

Interquartile Range (IQR)

Range between 25th and 75th percentiles; robust to extreme values.

Range

Difference between maximum and minimum values; simplest measure.

Median Absolute Deviation

Median of absolute deviations from median; robust central measure.

Coefficient of Variation (CV)

Relative measure: SD divided by mean; unitless, compares variability across datasets.

References

• Feller, W. "An Introduction to Probability Theory and Its Applications", Vol. 1, Wiley, 1968, pp. 123-130.
• Casella, G., Berger, R. L. "Statistical Inference", Duxbury Press, 2002, pp. 202-210.
• Rice, J. A. "Mathematical Statistics and Data Analysis", Duxbury Press, 2006, pp. 50-55.
• Ross, S. M. "Introduction to Probability Models", Academic Press, 2014, pp. 145-150.
• DeGroot, M. H., Schervish, M. J. "Probability and Statistics", Pearson Education, 2012, pp. 98-105.