Linear Algebra

Introduction to Linear Algebra

Linear algebra is one of the most widely applicable branches of mathematics. While algebra deals with single unknown quantities, linear algebra deals with collections of unknowns organized into vectors and matrices, and it studies the linear relationships between them.

The word "linear" refers to straight lines and flat planes — the simplest geometric objects. A linear equation in variables x₁, x₂, …, xₙ has the form:

where a₁, a₂, …, aₙ and b are constants. There are no squares, cubes, products of variables, or other nonlinear terms — just variables multiplied by constants and added together.

Linear algebra asks — and answers — fundamental questions such as:

• Does a system of linear equations have a solution? If so, is it unique?
• How can we efficiently represent and solve systems with hundreds or millions of unknowns?
• What happens when we apply a linear transformation to a geometric object?
• What are the "natural directions" associated with a matrix (eigenvectors)?

The subject was developed over centuries, with key contributions from mathematicians including Carl Friedrich Gauss (elimination methods), Arthur Cayley (matrix theory), Hermann Grassmann (vector spaces), and David Hilbert (abstract spaces). Today, linear algebra is typically the first mathematics course that moves from calculation to abstraction and proof.

Vectors

A vector is a mathematical object that has both magnitude (length) and direction. Geometrically, a vector is an arrow pointing from one location to another. Algebraically, a vector is an ordered list of numbers called components.

Notation and Representation

A vector in ℝⁿ (n-dimensional real space) is written as:

For example, the vector v = (3, 4) in ℝ² represents a point or an arrow from the origin to the point (3, 4) in the plane.

Common notations for vectors include boldface (v), an arrow above the letter (v⃗), or an underline. We'll use boldface throughout this lesson.

Vector Addition

Vectors of the same dimension are added component by component:

Geometrically, adding two vectors corresponds to placing them tip-to-tail and drawing the arrow from the start of the first to the end of the second (the parallelogram rule).

Scalar Multiplication

Multiplying a vector by a scalar (a real number) scales every component:

If c > 1, the vector stretches. If 0 < c < 1, it shrinks. If c < 0, it reverses direction and scales.

Magnitude (Length) of a Vector

The magnitude (or norm) of a vector v = (v₁, v₂, …, vₙ) is:

Unit Vectors

A unit vector has magnitude 1. To find the unit vector in the direction of v, divide by its magnitude:

The standard unit vectors in ℝ³ are i = (1, 0, 0), j = (0, 1, 0), and k = (0, 0, 1).

Dot Product (Scalar Product)

The dot product of two vectors is a scalar quantity defined as:

The dot product can also be expressed geometrically:

where θ is the angle between the two vectors.

Finding the Angle Between Vectors

Rearranging the geometric dot product formula gives:

Cross Product (Vector Product)

The cross product is defined only in ℝ³ and produces a new vector that is perpendicular to both input vectors:

The magnitude of the cross product equals the area of the parallelogram formed by the two vectors:

Matrices

A matrix is a rectangular array of numbers arranged in rows and columns. An m × n matrix has m rows and n columns. Matrices are the central data structure of linear algebra and provide a compact way to represent systems of equations, transformations, and data.

Matrix Notation

A general m × n matrix A is written as:

The entry in row i and column j is denoted aᵢⱼ.

Types of Matrices

• Row vector: A 1 × n matrix (a single row)
• Column vector: An m × 1 matrix (a single column)
• Square matrix: An n × n matrix (equal rows and columns)
• Zero matrix: All entries are 0
• Identity matrix (I): A square matrix with 1s on the main diagonal and 0s elsewhere
• Diagonal matrix: A square matrix where all non-diagonal entries are 0
• Symmetric matrix: A square matrix where A = Aᵀ (i.e., aᵢⱼ = aⱼᵢ for all i, j)
• Upper triangular: All entries below the main diagonal are 0
• Lower triangular: All entries above the main diagonal are 0

Matrix Addition and Scalar Multiplication

Matrices of the same dimensions are added entry by entry, and scalar multiplication scales every entry:

Matrix Multiplication

The product AB is defined when the number of columns in A equals the number of rows in B. If A is m × n and B is n × p, then AB is m × p:

Each entry of AB is the dot product of the corresponding row of A with the corresponding column of B.

Transpose

The transpose of an m × n matrix A, written Aᵀ, is the n × m matrix obtained by swapping rows and columns:

Key transpose properties:

• (Aᵀ)ᵀ = A
• (A + B)ᵀ = Aᵀ + Bᵀ
• (cA)ᵀ = cAᵀ
• (AB)ᵀ = BᵀAᵀ (note the reversal of order!)

Systems of Linear Equations

A system of linear equations is a collection of one or more linear equations involving the same set of variables. Linear algebra provides systematic methods for solving such systems efficiently, even when they involve thousands of variables.

Matrix Form of a System

Any system of linear equations can be written in matrix form as:

where A is the coefficient matrix, x is the vector of unknowns, and b is the vector of constants.

Augmented Matrix

To solve a system, we form the augmented matrix [A | b] by appending the constants column to the coefficient matrix:

Gaussian Elimination

Gaussian elimination is the fundamental algorithm for solving systems of linear equations. It uses three elementary row operations to systematically reduce the augmented matrix:

1. Row swap: Interchange two rows (Rᵢ ↔ Rⱼ)
2. Row scaling: Multiply a row by a nonzero constant (Rᵢ → cRᵢ)
3. Row replacement: Add a multiple of one row to another (Rᵢ → Rᵢ + cRⱼ)

Row Echelon Form (REF)

A matrix is in row echelon form if:

1. All rows consisting entirely of zeros are at the bottom
2. The first nonzero entry (called the pivot) in each nonzero row is to the right of the pivot in the row above
3. All entries below a pivot are zero

Reduced Row Echelon Form (RREF)

A matrix is in reduced row echelon form if it satisfies all REF conditions plus:

1. Each pivot is 1
2. Each pivot is the only nonzero entry in its column

Types of Solutions

A system of linear equations has exactly one of three possibilities:

• One unique solution: Every variable is a pivot variable (the system is consistent and determined)
• Infinitely many solutions: The system is consistent but has free variables (underdetermined)
• No solution: The system is inconsistent — a row of the form [0 0 … 0 | b] with b ≠ 0 appears

Determinants

The determinant is a scalar value associated with every square matrix. It encodes important information about the matrix: whether it's invertible, how it scales area/volume, and more. The determinant of matrix A is written det(A) or |A|.

2 × 2 Determinant

For a 2 × 2 matrix:

3 × 3 Determinant (Cofactor Expansion)

For a 3 × 3 matrix, we expand along the first row using cofactor expansion:

Properties of Determinants

• det(I) = 1 — the identity matrix has determinant 1
• det(Aᵀ) = det(A) — transposing doesn't change the determinant
• det(AB) = det(A) · det(B) — determinants multiply
• det(cA) = cⁿ det(A) — for an n × n matrix A
• Swapping two rows changes the sign of the determinant
• Multiplying a row by c multiplies the determinant by c
• Adding a multiple of one row to another does not change the determinant
• If any row (or column) is all zeros, the determinant is 0
• If two rows (or columns) are identical, the determinant is 0

Determinant and Invertibility

A square matrix A is invertible (nonsingular) if and only if det(A) ≠ 0. If det(A) = 0, the matrix is singular — it squashes space into a lower dimension, losing information irreversibly.

Matrix Inverse

An n × n matrix A is invertible if there exists a matrix A⁻¹ such that:

where I is the n × n identity matrix. The inverse "undoes" the action of A.

Inverse of a 2 × 2 Matrix

If A = [a b; c d] and det(A) = ad - bc ≠ 0, then:

Finding the Inverse Using Row Reduction

For larger matrices, augment A with the identity matrix and row reduce to RREF:

If the left side reduces to I, the right side is A⁻¹. If the left side cannot be reduced to I (you get a row of zeros), then A is not invertible.

Properties of the Inverse

• (A⁻¹)⁻¹ = A
• (AB)⁻¹ = B⁻¹A⁻¹ (note the reversal!)
• (Aᵀ)⁻¹ = (A⁻¹)ᵀ
KvkL0rjzSfsOPRI93hW13kWJcmEX7GDNRfa+301i3tWBSQbuJ+Ik3TY92iPxuaLFUmw7kXNAB1QFjwYJV3ajm1RxNiLrRKePhj3aLHZzEWbnG47YJOFDmdtmDqfl1xl+yn+MRYTOTcmRHfpVZ4mU/P/Y5udInU3zmKbZUYwULfRUEVuf9NtMzXlTfrZLVg2v/MLD4AEAAEa9OTXOz02BZB44uatqzcmzzCeC040CosYLmwD/lw99O1hqhrRC0MsM26UE/RzbYAIXFaRp8iLa3cbYmOiQKljOzkepFOHuW+hPyUspOJuoHtnCRZVOxmW+bwsL8GjbhAIiaIKFaz18aNhQfTOIM56Vg6lTayCHrwVDD7nPXEx93+60PDHdgt+NvzcZvZLib95RslhEqKPIdFAqGrX2PpPG6IfHaabrSfmeJTm21N75oh2Okr9BRNU/pwBJkfgigHiAP3KPnRk1+BaIzGQkp5hGzcYh1o1hwMa12M2EuqDbhDdXLNtFP4Ne3JRokv0MBzQ8wiSVD8LbUu5mlgaWrqRKV+6tNKuiiUEuH7UFH7fsMXSjCEF4F8pfyHEeGSGq5f8qur1xTbKUWl2ECIMuZBgC2cYlYJQ/AOR68ACc63PQCfOuiJP9Iv8VoID3ino4i+LjUloNblOJFvkr+QwiaMsnF3kKZw0BmaUd2TbK26boEoL0RLIxdiU2qByNEr1hX8pDfPd2mNt4vHKaaE+PYtckR52IVCLaoV7n2P7HT
• det(A⁻¹) = 1/det(A)
• (cA)⁻¹ = (1/c)A⁻¹

Cramer's Rule

Cramer's Rule provides an explicit formula for the solution of a system Ax = b when A is invertible. For each variable xᵢ:

where Aᵢ is the matrix formed by replacing the i-th column of A with b.

Vector Spaces

A vector space (over ℝ) is a set V together with two operations — addition and scalar multiplication — that satisfy ten axioms. Vector spaces abstract the familiar properties of arrows in the plane to much more general settings.

Axioms of a Vector Space

A set V with operations + (addition) and · (scalar multiplication) is a vector space if for all u, v, w ∈ V and scalars c, d ∈ ℝ:

1. u + v ∈ V (closure under addition)
2. u + v = v + u (commutativity)
3. (u + v) + w = u + (v + w) (associativity of addition)
4. There exists a zero vector 0 such that v + 0 = v
5. For each v, there exists -v such that v + (-v) = 0
6. cv ∈ V (closure under scalar multiplication)
7. c(u + v) = cu + cv (distributivity over vector addition)
8. (c + d)v = cv + dv (distributivity over scalar addition)
9. c(dv) = (cd)v (associativity of scalar multiplication)
10. 1v = v (multiplicative identity)
MuIkvoI5MXntMub8/VWcl7pGWn5O+Pkhy8++vIN9AO4AGj8HcS7fZevjGX1sPBKz5ZI3jEFjCGaD9rAGTi48dyYY3S7rlhetaewLHtZNMybjJK3xEbqfi8FLSf2QuNNf+wK76U5PiVCg96ZvTaCULeu2ob0WBG94MsGJYHyZ2LEqy0++Q3pXEr01+PPQlSVCpJytNgJFEumpq2tNsSxvQeMEFN3rP2PYqWBujTnlRSsi+qvVlH7ccXj83rIMtq999vA6ej85Z0GJPr8kwaNyObBRIp5RBOWAS/9+Tqxm+lqxiOYWQwiXFuXXEcHoB0dDM78+cUNMv6xDHNU2CXxdDxw4NkLja5WBPD7ZB033E9ZKBZQWDmM4ROvtEMiqzX/lEPFWmLvADAqAvwC6uEOAffOB8Ib62XDgS9e7CdKodzplCvPsC6Til2CInz2wKvxnC35DLOJH7OtSJam0jshik8vkRRYBM2Tw6Gy+Vie/B8/mTkupaDxravdgCfB4P0oOgNhjVPgazYLr9AUp6BlBAELXAvG3oiExlkrxaO//gPWDK1ripKlyAit0EzXmZFnWQlkokE+mDqBYAeih65xZaXolDUnmEsSP5bNzCIBK3E1/JOtpk5XaxAlfORnD5eDlgoV2aYRVuxsPjEHNj6DSMaxlMOlNfvypX87g5ASkJDVhEwPi/5c6JuZ2fH+YTdo4WONxiQmESmBwa2fx4Vs6DFsOkV47vfjKNBeNqHS2a804/lARm

Common examples of vector spaces include:

• ℝⁿ — the set of all n-tuples of real numbers
• Pₙ — the set of all polynomials of degree at most n
• M₂ₓ₂ — the set of all 2 × 2 matrices with real entries
• C[a,b] — the set of all continuous functions on [a,b]

Subspaces

A subspace of a vector space V is a nonempty subset W ⊆ V that is itself a vector space under the same operations. To verify W is a subspace, check three conditions:

1. The zero vector is in W
2. W is closed under addition: if u, v ∈ W, then u + v ∈ W
3. W is closed under scalar multiplication: if v ∈ W and c ∈ ℝ, then cv ∈ W

Linear Combinations and Span

A linear combination of vectors v₁, v₂, …, vₖ is any sum of the form:

where c₁, c₂, …, cₖ are scalars.

The span of a set of vectors is the set of all possible linear combinations of those vectors. It is the "smallest" subspace containing all the given vectors:

Linear Independence

A set of vectors {v₁, v₂, …, vₖ} is linearly independent if the only solution to:

is c₁ = c₂ = … = cₖ = 0. Otherwise, the set is linearly dependent, meaning at least one vector can be expressed as a linear combination of the others.

Basis and Dimension

A basis for a vector space V is a set of vectors that is:

1. Linearly independent
2. Spans V

A basis is a minimal spanning set — every vector in V can be written uniquely as a linear combination of the basis vectors.

The dimension of a vector space is the number of vectors in any basis (all bases have the same size).

Linear Transformations

A linear transformation is a function T: V → W between two vector spaces that preserves the operations of addition and scalar multiplication:

Equivalently, a function is linear if and only if T(c₁v₁ + c₂v₂) = c₁T(v₁) + c₂T(v₂) for all vectors and scalars.

Linear transformations include familiar operations like:

• Rotations — spinning vectors around the origin
• Reflections — flipping vectors across a line or plane
• Scaling — stretching or compressing vectors
• Projections — "flattening" vectors onto a subspace
• Differentiation — T(f) = f' is a linear transformation on the space of polynomials

Matrix Representation

Every linear transformation T: ℝⁿ → ℝᵐ can be represented as multiplication by an m × n matrix A:

The matrix A is found by applying T to each standard basis vector and using the results as columns:

Common Transformation Matrices in ℝ²

Here are standard transformation matrices for geometric operations:

• Rotation by angle θ: [cos θ -sin θ; sin θ cos θ]
• Reflection across x-axis: [1 0; 0 -1]
• Reflection across y-axis: [-1 0; 0 1]
• Reflection across y = x: [0 1; 1 0]
• Scaling by factors a, b: [a 0; 0 b]
• Horizontal shear: [1 k; 0 1]
• Projection onto x-axis: [1 0; 0 0]

Kernel and Image

Two fundamental subspaces are associated with every linear transformation T: V → W:

The kernel (or null space) of T is the set of all vectors that T maps to zero:

The image (or range) of T is the set of all possible output vectors:

Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors are among the most important concepts in linear algebra. They reveal the "natural" behavior of a linear transformation — the directions that are merely scaled, not rotated.

Definition

Let A be an n × n matrix. A nonzero vector v is an eigenvector of A if:

for some scalar λ. The scalar λ is called the corresponding eigenvalue. In other words, multiplying v by A simply scales v by the factor λ — the direction of v is preserved (or reversed if λ < 0).

The Characteristic Equation

To find eigenvalues, we rearrange Av = λv:

For a nonzero solution v to exist, the matrix (A - λI) must be singular, which means:

This is the characteristic equation. The left side, when expanded, is a polynomial of degree n in λ called the characteristic polynomial.

Key Properties of Eigenvalues

• The sum of eigenvalues equals the trace of A: λ₁ + λ₂ + … + λₙ = tr(A) = a₁₁ + a₂₂ + … + aₙₙ
• The product of eigenvalues equals the determinant: λ₁ · λ₂ · … · λₙ = det(A)
• A is singular (non-invertible) if and only if 0 is an eigenvalue
• Symmetric matrices always have real eigenvalues
• Eigenvectors corresponding to distinct eigenvalues are linearly independent

Diagonalization

An n × n matrix A is diagonalizable if it can be written as:

where D is a diagonal matrix of eigenvalues and P is the matrix whose columns are the corresponding eigenvectors. A matrix is diagonalizable if and only if it has n linearly independent eigenvectors.

Applications of Linear Algebra

Linear algebra is not just an abstract mathematical subject — it is the computational engine behind many of the technologies and scientific methods that shape the modern world. Here are some of its most impactful applications.

Computer Graphics

Every image, animation, and 3D model you see on a screen is rendered using linear algebra. Transformations of objects — rotation, scaling, translation, and perspective projection — are all represented as matrix operations.

In 3D graphics, homogeneous coordinates extend ℝ³ to ℝ⁴ so that translations (which are not linear) can also be represented as matrix multiplication. A point (x, y, z) becomes (x, y, z, 1), and all transformations become 4 × 4 matrices:

Composing multiple transformations is simply multiplying matrices together. The GPU in your computer is essentially a massively parallel linear algebra engine.

Data Science and Machine Learning

Data science uses linear algebra at every level. Datasets are naturally represented as matrices (rows = samples, columns = features), and most machine learning algorithms are built on linear algebraic operations:

• Linear Regression: Finds the best-fit line (or hyperplane) by solving a system of equations. The famous normal equation is x = (AᵀA)⁻¹Aᵀb.
• Principal Component Analysis (PCA): Uses eigenvalues and eigenvectors of the covariance matrix to identify the most important directions in high-dimensional data, enabling dimensionality reduction.
• Neural Networks: Every layer in a neural network is essentially a linear transformation (matrix multiplication) followed by a nonlinear activation function.
• Recommender Systems: Services like Netflix and Spotify use matrix factorization to predict user preferences.

Google PageRank

One of the most famous applications of linear algebra is Google's original PageRank algorithm, which ranked web pages by importance. The key insight is modeling the web as a Markov chain:

1. Represent the web as a directed graph where pages are nodes and links are edges
2. Construct a transition matrix M where Mᵢⱼ = 1/Lⱼ if page j links to page i (Lⱼ = number of outgoing links from j)
3. The PageRank vector r is the eigenvector of M corresponding to eigenvalue 1:

In practice, a damping factor d ≈ 0.85 is used:

This is solved iteratively (power iteration), and pages with higher PageRank values appear higher in search results.

Least Squares Approximation

When a system Ax = b has no exact solution (more equations than unknowns, i.e., the system is overdetermined), we seek the least squares solution — the vector x̂ that minimizes the total squared error ‖Ax - b‖².

The solution satisfies the normal equations:

If AᵀA is invertible:

Quantum Mechanics

Quantum mechanics is fundamentally built on linear algebra. The state of a quantum system is represented as a vector in a complex vector space (Hilbert space), and physical observables (like energy, momentum, position) are represented as Hermitian matrices. Measurements correspond to eigenvalues, and the system's state collapses to the corresponding eigenvector after measurement.

Economics and Network Analysis

Linear algebra appears in economics through input-output models (Leontief model), where the relationships between industries are captured in a matrix. In network analysis, matrices encode connections between nodes, and their eigenvalues reveal properties like connectivity, clustering, and influence.

Orthogonality and Projections

Two vectors are orthogonal (perpendicular) if their dot product is zero. Orthogonality is one of the most powerful concepts in linear algebra — it simplifies computations, enables decompositions, and underpins algorithms from signal processing to machine learning.

Orthogonal and Orthonormal Sets

Orthonormal bases are especially convenient: to express any vector x in terms of orthonormal basis {u₁, …, uₙ}, you simply compute:

No matrix inversion required!

Orthogonal Projection

The projection of vector b onto a subspace W is the closest vector in W to b. If W = span(a), the projection is:

The error (or residual) e = b − proj_a(b) is orthogonal to a. This is the geometric foundation of least-squares regression.

The Gram-Schmidt Process

Given any linearly independent set {v₁, …, vₖ}, the Gram-Schmidt process produces an orthonormal set {u₁, …, uₖ} spanning the same subspace:

Each step subtracts the projections onto all previously computed orthonormal vectors, then normalizes.

QR Decomposition

Gram-Schmidt applied to the columns of a matrix A produces the QR decomposition:

where Q is an orthogonal matrix (Qᵀ = Q⁻¹) and R is upper triangular. QR decomposition is numerically stable and is the basis of modern algorithms for computing eigenvalues (the QR algorithm).

Orthogonal Matrices

A square matrix Q is orthogonal if QᵀQ = QQᵀ = I, meaning Q⁻¹ = Qᵀ. Orthogonal matrices preserve lengths and angles — they represent rotations and reflections. Key properties:

• ‖Qx‖ = ‖x‖ for all x (length-preserving)
• det(Q) = ±1
• Eigenvalues have absolute value 1
• Product of orthogonal matrices is orthogonal
nC7fwql4dijn52ujkeIPUsWISVYdrw2fYAkkO+A97F3TknFJvaMJIsOMdgInLx05DSt0Ow8Vk5ceh2JAOMmCZHlEL5SwZw/6f7Mo1gCHIFxFdED20qYwbWUTDmb83TTuxhlLMrbsNOUhQEpZbFndtCfvwgS1MHEVjr+uCkmFeD1Q11H3FxgEIqwC/kk/yWoyH3JSHvJaYjqpiR1hZnSEuueyfhbszNxf1YIpXWSxTgLBrfJKs8YbMv2QPB/iOB9sKMfjVD7hOJjZq8yLhvEDYJp6G1wn2hokL0hL1uILmS1D5k0TM6lRocHIeUyvT9/6cexa0ArHCN3s6QAtebKQaKAbdKXpub2nLLMEnITEkVtafTaIZwlb2qtAWALIq+RNxpyyB1/YJxEFPgCi0482z747WQZZk/S+HhoDPUwhrf27oLDOwA5moRC3+uS6ObZytP3zrq6IWp8ap20r933LE7opHAzAUVkb9+p4suqyWwCLvgm//xiyRlx/9ZkjQfszyJWgLtriIs64fAgxdrod0fFvDLzX6SmrfZpekEHJkMqjUgOj36ttKGx8+C7b4bKKBgwQbuk1SxQRmzHj7BCFGpjcGXVDfdersH3Cc2x/jG2GXsbNfGFJ66e5faOdA0ECjdCMSgKb62G3l2TsDlBfrcQB9n/mwB5FHsBnwZSdd/3vYplXVnq6/P+OZ8gYPLnypXp9vb5wa8HEa/k4HG/qdAs9256jASY+kUbTDIT7Z5CCGjHvJ

Singular Value Decomposition (SVD)

The Singular Value Decomposition is arguably the most important matrix factorization in applied mathematics. Every matrix — of any shape, any rank — has an SVD.

The SVD Theorem

• U (m × m): Orthogonal matrix — columns are the left singular vectors (eigenvectors of AAᵀ)
• Σ (m × n): Diagonal matrix of singular values σ₁ ≥ σ₂ ≥ ⋯ ≥ 0 (square roots of eigenvalues of AᵀA)
• Vᵀ (n × n): Orthogonal matrix — rows are the right singular vectors (eigenvectors of AᵀA)

Geometric Interpretation

Every linear transformation can be decomposed into three steps:

1. Vᵀ: Rotate the input space (align with principal axes)
2. Σ: Scale along each axis by the singular values
3. U: Rotate the output space

The singular values tell you how much the matrix stretches in each direction. The rank of A equals the number of nonzero singular values.

Low-Rank Approximation (Eckart-Young Theorem)

The SVD provides the best rank-k approximation of a matrix. Keep only the k largest singular values and their corresponding vectors:

A_k is the closest rank-k matrix to A (in both Frobenius and spectral norms). This is the mathematical foundation of:

• Image compression: Store only the top k singular values/vectors instead of the full image matrix
• Dimensionality reduction (PCA): Principal Component Analysis is SVD applied to centered data
• Latent Semantic Analysis: Discover hidden topics in text document collections
• Recommender systems: Netflix Prize-winning algorithms use SVD for collaborative filtering
• Noise reduction: Small singular values often correspond to noise; truncating them denoises the data

The Pseudoinverse

For any matrix A (even non-square or singular), the Moore-Penrose pseudoinverse A⁺ can be computed via SVD:

where Σ⁺ is formed by taking the reciprocal of each nonzero singular value. The pseudoinverse gives the least-squares solution to Ax = b when no exact solution exists.