Vectors & Matrices

Vectors

A vector is an ordered list of numbers representing magnitude and direction. In ℝⁿ, vectors can represent points, displacements, or abstract data.

The magnitude formula generalizes the distance formula from coordinate geometry. The dot product connects to trigonometric functions via the angle between vectors.

Matrices

A matrix is a rectangular array of numbers. An m×n matrix has m rows and n columns.

Special matrices: diagonal, upper/lower triangular, symmetric, orthogonal (QᵀQ = I). The identity matrix I acts like 1 in matrix multiplication — similar to how 1 is the multiplicative identity.

Matrix Operations

1. Addition: (A + B)ᵢⱼ = Aᵢⱼ + Bᵢⱼ (same dimensions required)
2. Scalar multiplication: (cA)ᵢⱼ = c·Aᵢⱼ
3. Matrix multiplication: (AB)ᵢⱼ = Σₖ Aᵢₖ·Bₖⱼ (inner dimensions must match)
4. Inverse: AA⁻¹ = A⁻¹A = I (only for square, non-singular matrices)

Solving Linear Systems

Linear systems Ax = b can be solved via:

• Gaussian elimination: Row reduce to echelon form
• Matrix inverse: x = A⁻¹b (when A is invertible)
• Cramer's rule: xᵢ = det(Aᵢ)/det(A)

These generalize the methods for solving systems of equations to any number of variables. In regression analysis, the normal equations have the form b = (XᵀX)⁻¹Xᵀy.

Determinants

The determinant encodes whether a matrix is invertible (det ≠ 0), the scaling factor of the associated transformation, and the signed volume of the parallelepiped formed by column vectors.