Eigenvalues & Eigenvectors

1. Introduction

Most linear transformations mix coordinates together in complicated ways: a matrix can rotate, stretch, shear, and reflect all at once. Eigenvalues and eigenvectors cut through this complexity by finding the special directions in which a transformation acts as nothing more than simple scaling. Along an eigenvector, the matrix neither rotates nor shears — it merely stretches (or shrinks, or flips) the vector by a factor called the eigenvalue. These directions are the natural axes of the transformation, the coordinate system in which the matrix becomes diagonal and its behavior becomes transparent.

The importance of eigentheory is hard to overstate. It diagonalizes matrices so that powers $A^k$ become trivial; it solves systems of linear differential equations; it underlies Principal Component Analysis in statistics, the PageRank algorithm, vibration analysis in engineering, and the stationary states of quantum mechanics. Whenever a system evolves by repeated application of a linear rule, the eigenvalues govern its long-term growth, decay, or oscillation, and the eigenvectors describe its fundamental modes.

This article builds the theory carefully: the eigen-equation and its geometric meaning, the characteristic polynomial, eigenspaces, algebraic and geometric multiplicity, diagonalizability, the Spectral Theorem for symmetric matrices, and applications to matrix powers and differential equations. Along the way we derive why the characteristic equation works and prove the trace/determinant relations.

Eigenvalues answer the fundamental question: in which directions does a linear transformation act by simple scaling? This decomposition is the key to understanding dynamical systems, vibrating structures, quantum mechanical observables, and data compression via Principal Component Analysis. The spectral theorem for symmetric matrices is one of the most important results in all of applied mathematics.

2. Core Concepts

2.1 The Eigen-Equation

Given $A\in\mathbb{R}^{n\times n}$, a nonzero vector $\mathbf{v}$ is an eigenvector if $A\mathbf{v} = \lambda\mathbf{v}, \qquad \mathbf{v}\ne\mathbf{0},$ where $\lambda$ is the eigenvalue. Geometrically, the line through $\mathbf{v}$ is left invariant by $A$; vectors on it are scaled by $\lambda$ (reversed if $\lambda<0$).

2.2 The Characteristic Equation

Rewrite as $(A-\lambda I)\mathbf{v}=\mathbf{0}$. A nonzero solution exists iff $A-\lambda I$ is singular, i.e. $\det(A-\lambda I) = 0.$ The characteristic polynomial $p(\lambda)=\det(A-\lambda I)$ has degree $n$; its roots are the eigenvalues (over $\mathbb{C}$, always $n$ counting multiplicity).

2.3 Eigenspaces, Multiplicity, and Diagonalizability

For each eigenvalue $\lambda$, the eigenspace $E_\lambda=\ker(A-\lambda I)$ is a subspace. Two notions of multiplicity:

• Algebraic multiplicity: multiplicity as a root of $p(\lambda)$.
• Geometric multiplicity: $\dim E_\lambda$.

Always $1\le$ geometric $\le$ algebraic. $A$ is diagonalizable iff geometric = algebraic for every eigenvalue — equivalently, $A$ has $n$ linearly independent eigenvectors. Distinct eigenvalues guarantee diagonalizability (though not necessary).

2.4 Diagonalization

When $A$ has $n$ independent eigenvectors, form $P$ (eigenvectors as columns) and $D$ (eigenvalues on diagonal): $A = PDP^{-1}, \qquad A^k = PD^kP^{-1}.$ This also defines matrix functions: $e^{A}=Pe^{D}P^{-1}$, central to solving $\mathbf{x}'=A\mathbf{x}$.

2.5 Trace and Determinant Relations

Comparing $p(\lambda)=\prod_i(\lambda_i-\lambda)$ with $\det(A-\lambda I)$: $\operatorname{tr}(A) = \sum_{i=1}^n \lambda_i, \qquad \det(A) = \prod_{i=1}^n \lambda_i.$

Proof sketch. The coefficient of $\lambda^{n-1}$ in $\det(A-\lambda I)$ is $-(\sum a_{ii})=-\operatorname{tr}(A)$, and the constant term is $\det(A)$. Matching with the factored form gives the relations.

2.6 The Spectral Theorem

If $A$ is real and symmetric ($A=A^{T}$), then:

• all eigenvalues are real;
• eigenvectors for distinct eigenvalues are orthogonal;
• $A$ is orthogonally diagonalizable: $A = QDQ^{T}$ with $Q$ orthogonal ($Q^{T}=Q^{-1}$).

Proof intuition for real eigenvalues. If $A\mathbf{v}=\lambda\mathbf{v}$, then $\lambda\|\mathbf{v}\|^2 = \mathbf{v}^{T}A\mathbf{v} = (A\mathbf{v})^{T}\mathbf{v} = \bar\lambda\|\mathbf{v}\|^2$, so $\lambda=\bar\lambda$ is real. For orthogonality of eigenvectors with distinct eigenvalues: $\lambda_1\mathbf{v}_1^{T}\mathbf{v}_2 = (A\mathbf{v}_1)^{T}\mathbf{v}_2 = \mathbf{v}_1^{T}A\mathbf{v}_2 = \lambda_2\mathbf{v}_1^{T}\mathbf{v}_2$, so $(\lambda_1-\lambda_2)\mathbf{v}_1^{T}\mathbf{v}_2=0$ forces orthogonality.

This theorem underpins PCA, quadratic-form classification, and the principal-axis theorem.

2.7 Applications to Differential Equations

For $\mathbf{x}'=A\mathbf{x}$, solutions are linear combinations of $e^{\lambda_i t}\mathbf{v}_i$. The sign of $\lambda_i$ determines growth ($\lambda>0$), decay ($\lambda<0$), or oscillation (complex $\lambda$). The dominant eigenvalue governs long-term behavior.