Definition of Independence
Conceptual Meaning
Independence: absence of influence between two events. Occurrence of one event does not alter probability of the other.
Formal Mathematical Definition
Events A and B are independent if and only if:
P(A ∩ B) = P(A) × P(B)Where P(A ∩ B) is joint probability, P(A), P(B) are individual probabilities.
Extension to Multiple Events
Events A1, A2, ..., An are mutually independent if:
P(A_{i_1} ∩ A_{i_2} ∩ ... ∩ A_{i_k}) = ∏_{j=1}^k P(A_{i_j})for every subset {i1, ..., ik} ⊆ {1,...,n}, 1 ≤ k ≤ n.
Independent Events
Definition
Two events are independent if knowledge of one event’s occurrence does not change the probability of the other.
Equivalent Conditions
Equivalence between:
- P(A ∩ B) = P(A)P(B)
- P(A | B) = P(A) if P(B) > 0
- P(B | A) = P(B) if P(A) > 0
Visual Representation
Venn diagrams: independence not visually represented by disjointness; events can overlap yet be independent.
Conditional Probability and Independence
Conditional Probability Definition
For events A and B, with P(B) > 0:
P(A | B) = P(A ∩ B) / P(B)Independence Criterion Using Conditional Probability
Independence implies:
P(A | B) = P(A)Interpretation
Knowing B occurs does not update probability of A if independent.
Multiplication Rule for Independent Events
General Multiplication Rule
For any events A, B:
P(A ∩ B) = P(A | B) × P(B)Independent Events Simplification
If A and B independent:
P(A ∩ B) = P(A) × P(B)Extension to Multiple Events
For mutually independent events A1, ..., An:
P(∩_{i=1}^n A_i) = ∏_{i=1}^n P(A_i)Dependent Events and Contrast
Definition of Dependence
Events A and B dependent if:
P(A ∩ B) ≠ P(A) × P(B)Effect on Conditional Probability
If dependent:
P(A | B) ≠ P(A)Examples
Drawing cards without replacement, weather affecting traffic, medical test outcomes.
Independence of Random Variables
Definition
Random variables X and Y independent if for all x, y:
P(X ≤ x, Y ≤ y) = P(X ≤ x) × P(Y ≤ y)Joint and Marginal Distributions
Joint cumulative distribution function (CDF) factorizes into product of marginals.
Extension to Multiple Variables
Mutual independence requires joint CDF equals product of all marginal CDFs.
Properties of Independent Events
Symmetry
If A independent of B, then B independent of A.
Complement Independence
If A and B independent, then A and Bc, Ac and B, Ac and Bc are also independent.
Independence and Unions
Independence generally not preserved under unions or intersections beyond original events.
Examples of Independence
Coin Tosses
Outcomes of fair coin tosses independent; previous toss does not affect next.
Dice Rolls
Rolls of fair dice independent; probability of one outcome unrelated to another.
Card Draws with Replacement
Drawing cards with replacement ensures independence; without replacement induces dependence.
| Scenario | Independence | Reason |
|---|---|---|
| Two coin tosses | Independent | Outcome of one does not affect other |
| Two card draws without replacement | Dependent | Second draw changes card pool |
| Two dice rolls | Independent | No influence between rolls |
Testing for Independence
Empirical Testing
Compare observed joint frequencies with product of marginal frequencies.
Chi-Square Test
Statistical test for independence in contingency tables.
Correlation and Independence
Zero correlation does not imply independence except for jointly normal variables.
Applications of Independence
Model Simplification
Assuming independence reduces complexity in probabilistic models and calculations.
Bayesian Networks
Independence assumptions define network structure and conditional dependencies.
Reliability Engineering
Independent failure events simplify reliability computations.
| Application | Role of Independence |
|---|---|
| Bayesian inference | Conditional independence enables factorization of joint probabilities |
| Quality control | Independent defect occurrences simplify defect rate calculations |
| Machine learning | Feature independence assumptions improve model tractability |
Common Misconceptions
Independence vs. Mutual Exclusivity
Exclusive events cannot be independent unless one has zero probability.
Zero Correlation ≠ Independence
Correlation measures linear relationship only; independence is stronger condition.
Independence is Not Always Symmetric in Conditionals
Conditional independence may hold in one direction, not necessarily symmetric.
Summary
Independence is core to probability theory: events do not affect each other's likelihoods. Defined mathematically by product rule. Crucial in modeling, simplifying analyses, and understanding relationships between random phenomena.
References
- Ross, S. M. "Introduction to Probability Models." Academic Press, 11th edition, 2014, pp. 45-78.
- Grimmett, G., and Stirzaker, D. "Probability and Random Processes." Oxford University Press, 3rd edition, 2001, pp. 92-120.
- Billingsley, P. "Probability and Measure." Wiley, 3rd edition, 1995, pp. 65-88.
- Feller, W. "An Introduction to Probability Theory and Its Applications." Wiley, Vol. 1, 3rd edition, 1968, pp. 110-135.
- Casella, G., and Berger, R. L. "Statistical Inference." Duxbury, 2nd edition, 2002, pp. 230-255.