Definition
Matrix Rank
Rank: maximum number of linearly independent rows or columns in a matrix. Equivalently: dimension of column space or row space. Measures matrix's non-degeneracy and solution characteristics of associated linear systems.
Formal Definition
Given matrix A ∈ Mm×n(F), rank(A) = dim(span of columns) = dim(span of rows). Rank is invariant under elementary row operations.
Significance
Rank quantifies the degree of linear independence. Rank zero: zero matrix. Full rank: rank = min(m,n). Determines invertibility for square matrices.
Row Rank and Column Rank
Definitions
Row rank: dimension of row space (span of rows). Column rank: dimension of column space (span of columns).
Equivalence
Theorem: row rank = column rank for any matrix over a field. Fundamental result underpinning rank definition.
Proof Sketch
Proof: use elementary row operations to reach echelon form. Pivot positions count both row and column ranks. Duality of row and column spaces.
Properties of Rank
Basic Properties
Rank(A) ≤ min(m,n). Rank(AB) ≤ min(rank(A), rank(B)). Rank(A + B) ≤ rank(A) + rank(B).
Invariance
Rank invariant under multiplication by invertible matrices: rank(PAQ) = rank(A) if P,Q invertible.
Rank and Transpose
Rank(A) = Rank(AT). Row and column ranks equal.
Methods of Calculation
Gaussian Elimination
Use row operations to convert matrix to row echelon form. Count nonzero rows → rank.
Determinants and Minors
Rank = largest order of nonzero minors. Search for largest k×k submatrix with nonzero determinant.
Singular Value Decomposition (SVD)
Rank = number of nonzero singular values. Numerically stable method in applied contexts.
Rank-Revealing QR Factorization
QR with column pivoting identifies numerical rank in approximate computations.
Computational Complexity
Gaussian elimination: O(mn2). SVD: O(mn2 + n3).
Rank and Linear Independence
Column Rank and Independence
Rank equals maximal number of linearly independent columns. Dependent columns do not increase rank.
Row Rank and Independence
Rank equals maximal number of linearly independent rows. Rows form basis for row space.
Relation to Basis
Rank = dimension of column space = size of minimal generating set for range of associated linear transformation.
Rank Theorem
Statement
For linear transformation T: V → W, dim(V) = rank(T) + nullity(T). Rank-nullity theorem relates rank to kernel dimension.
Matrix Form
For A ∈ Mm×n, n = rank(A) + nullity(A). Nullity is dimension of null space (solutions to Ax = 0).
Applications
Determines solvability of linear systems: full rank implies unique solution if consistent.
Rank of Special Matrices
Diagonal and Identity Matrices
Rank of diagonal matrix = number of nonzero diagonal entries. Identity matrix rank = n (full rank).
Symmetric Matrices
Rank equal to number of nonzero eigenvalues. Symmetric matrices diagonalizable.
Zero and Sparse Matrices
Zero matrix rank = 0. Sparse matrices: rank depends on pattern of nonzero entries and their linear independence.
Rank and Nullity
Definitions
Nullity: dimension of null space (kernel) of matrix or linear transformation.
Interdependence
Rank + nullity = number of columns (dimension of domain). Fundamental linear algebra identity.
Geometric Interpretation
Rank: dimension of image. Nullity: dimension of kernel. Together span domain space.
Applications of Rank
Solving Linear Systems
Rank determines existence and uniqueness of solutions. Full rank matrices invertible.
Matrix Factorizations
Rank guides LU, QR, SVD factorization usage and interpretation.
Data Science and Statistics
Rank used in PCA, multicollinearity diagnosis, dimensionality reduction.
Control Theory
Rank conditions for controllability and observability of systems.
Examples
Example 1: Full Rank Matrix
Matrix: 3×3 identity matrix. Rank: 3. All rows and columns linearly independent.
Example 2: Rank Deficient Matrix
Matrix with two identical rows or columns. Rank less than min(m,n), indicating dependency.
Example 3: Rank via Minors
Find largest nonzero minor in 3×3 matrix. If largest minor order is 2, rank = 2.
Example 4: Rank and Nullity
Matrix A ∈ M2×3 with rank 2 implies nullity 1 (3 − 2 = 1).
Common Misconceptions
Rank Equals Number of Nonzero Elements
False. Rank depends on linear independence, not just count of nonzero entries.
Row Rank Different from Column Rank
False. Always equal over fields; row rank = column rank.
Zero Determinant Implies Zero Rank
False. Zero determinant means rank < n for square matrix; rank may be positive.
Advanced Topics
Rank over Rings and Modules
Rank generalizes with caution when base is not field. Rank concepts differ in module theory.
Numerical Rank
Approximate rank in numerical analysis using tolerance thresholds on singular values.
Rank Deficiency and Perturbations
Small perturbations can change rank in numerical matrices; stability analysis important.
Rank and Tensor Decompositions
Extension of rank concept to tensors: CP rank, multilinear rank, with distinct properties.
References
- Gilbert Strang, Introduction to Linear Algebra, Wellesley-Cambridge Press, 5th Ed., 2016, pp. 120-150.
- Steven J. Leon, Linear Algebra with Applications, Pearson, 9th Ed., 2015, pp. 200-230.
- Sheldon Axler, Linear Algebra Done Right, Springer, 3rd Ed., 2015, pp. 75-110.
- Gene H. Golub and Charles F. Van Loan, Matrix Computations, Johns Hopkins University Press, 4th Ed., 2013, pp. 50-90.
- David C. Lay, Linear Algebra and Its Applications, Pearson, 5th Ed., 2015, pp. 160-195.