Inner Product Spaces

Definition and Basic Properties

Inner Product

Definition: An inner product on a vector space V over field 𝔽 (ℝ or ℂ) is a function <·,·> : V×V → 𝔽 satisfying linearity, conjugate symmetry, and positive-definiteness.

Properties

Linearity: <ax + by, z> = a<x, z> + b<y, z> for scalars a,b ∈ 𝔽, vectors x,y,z ∈ V.

Conjugate symmetry: <x, y> = overline(<y, x>).

Positive-definiteness: <x, x> ≥ 0 with equality iff x = 0.

Consequences

Inner product induces norm and metric. Enables geometric notions: length, angle, orthogonality.

For all x, y, z ∈ V, a, b ∈ 𝔽:<x, y + z> = <x, y> + <x, z><x, a y> = a <x, y><x, y> = overline(<y, x>)<x, x> ≥ 0; <x, x> = 0 ⇔ x = 0

Examples of Inner Product Spaces

Euclidean Space ℝⁿ

Standard inner product: <x, y> = Σ xᵢ yᵢ for vectors x,y ∈ ℝⁿ.

Complex Space ℂⁿ

Inner product: <x, y> = Σ xᵢ overline(yᵢ), conjugate linear in second argument.

Function Spaces

Space L²([a,b]): <f, g> = ∫ₐᵇ f(t) overline(g(t)) dt, square-integrable functions.

Norm Induced by Inner Product

Definition

Norm: ||x|| = √<x, x>. Satisfies positivity, scalability, triangle inequality.

Properties

||x|| ≥ 0; ||x|| = 0 iff x=0. Homogeneity: ||a x|| = |a| ||x||. Triangle: ||x + y|| ≤ ||x|| + ||y||.

Metric Structure

Distance d(x,y) = ||x - y||. Turns V into metric space, enables convergence, continuity.

Norm induced by inner product:For x ∈ V,||x|| = sqrt(<x, x>)Properties:1. ||x|| ≥ 0 and ||x|| = 0 ⇔ x = 02. ||a x|| = |a| ||x|| for all scalars a3. ||x + y|| ≤ ||x|| + ||y|| (triangle inequality)

Orthogonality and Orthogonal Complements

Orthogonal Vectors

Definition: x ⟂ y if <x, y> = 0. Orthogonality generalizes perpendicularity.

Orthogonal Sets

Set {vᵢ} orthogonal if <vᵢ, vⱼ> = 0 for i ≠ j. Orthogonal sets are linearly independent.

Orthogonal Complement

For subset S ⊆ V, S⊥ = {x ∈ V : <x, s> = 0 ∀ s ∈ S}. Closed under addition and scalar multiplication.

Cauchy-Schwarz Inequality

Statement

|<x, y>| ≤ ||x|| · ||y|| for all x,y ∈ V.

Equality Condition

Equality iff x and y are linearly dependent: x = α y or y = β x for some scalar α, β.

Implications

Establishes inner product continuity. Basis for triangle inequality in normed spaces.

Cauchy-Schwarz inequality:For all x, y ∈ V,|<x, y>| ≤ ||x|| · ||y||Equality ⇔ x and y linearly dependent

Polarization Identity

Purpose

Expresses inner product in terms of norm alone. Enables recovery of <x, y> from ||·||.

Formulas

Real case: <x, y> = ¼ (||x + y||² - ||x - y||²).

Complex case: <x, y> = ¼ (||x + y||² - ||x - y||² + i||x + i y||² - i||x - i y||²).

Significance

Shows equivalence of norms induced by inner products and inner products themselves.

Orthonormal Bases and Gram-Schmidt Process

Orthonormal Set

Set {eᵢ} orthonormal if <eᵢ, eⱼ> = δᵢⱼ (Kronecker delta), i.e., vectors unit length and mutually orthogonal.

Orthonormal Basis

Basis that is orthonormal. Simplifies coordinates: x = Σ <x, eᵢ> eᵢ.

Gram-Schmidt Orthogonalization

Algorithm to convert linearly independent set {v₁, ..., vₙ} into orthonormal set {e₁, ..., eₙ}.

Gram-Schmidt process:Input: linearly independent {v₁,...,vₙ}Set e₁ = v₁ / ||v₁||For k = 2 to n: uₖ = vₖ - Σ_{j=1}^{k-1} <vₖ, eⱼ> eⱼ eₖ = uₖ / ||uₖ||Output: orthonormal set {e₁,...,eₙ}

Orthogonal Projections

Projection onto Subspace

For subspace W ⊆ V, orthogonal projection P: V → W satisfies P² = P, P self-adjoint, and image(P) = W.

Formula for Projection

Given orthonormal basis {eᵢ} of W, P(x) = Σ <x, eᵢ> eᵢ.

Properties

Minimizes distance: ||x - P(x)|| = inf_{w ∈ W} ||x - w||. Projection is linear, idempotent, and symmetric.

Linear Transformations in Inner Product Spaces

Adjoint Operator

For T: V → V linear, adjoint T* defined by <T x, y> = <x, T* y> ∀ x,y ∈ V.

Self-Adjoint Operators

Operator T with T = T*. Spectrum real, diagonalizable with orthonormal eigenbasis.

Unitary and Orthogonal Operators

Operators preserving inner product: <>T x, T y> = <x, y>. Unitary if complex, orthogonal if real.

Adjoint definition:For all x, y ∈ V,<T x, y> = <x, T* y>Self-adjoint: T = T*Unitary: <T x, T y> = <x, y>

Hilbert Spaces and Completeness

Definition

Hilbert space: inner product space complete with respect to norm induced by inner product.

Examples

ℓ² space of square-summable sequences, L² spaces of square-integrable functions are Hilbert spaces.

Importance

Framework for functional analysis, quantum mechanics, signal processing. Completeness enables limit operations.

Applications of Inner Product Spaces

Fourier Analysis

Decomposition of functions into orthonormal bases of trigonometric functions. Parseval's identity.

Quantum Mechanics

State spaces as complex Hilbert spaces. Inner product gives probability amplitudes.

Signal Processing

Projection onto basis functions for filtering, noise reduction, compression.

Machine Learning

Kernel methods use inner products in feature spaces for classification and regression.

References

• Halmos, P.R., Introduction to Hilbert Space and the Theory of Spectral Multiplicity, AMS, 1951, pp. 1–234.
• Axler, S., Linear Algebra Done Right, Springer, 3rd ed., 2015, pp. 1–350.
• Rudin, W., Functional Analysis, McGraw-Hill, 2nd ed., 1991, pp. 1–416.
• Lax, P.D., Linear Algebra and Its Applications, Wiley, 2nd ed., 2007, pp. 1–582.
• Conway, J.B., A Course in Functional Analysis, Springer, 2nd ed., 1990, pp. 1–470.