Linear Algebra

A vector can represent an observation or a set of parameters; a matrix represents a linear transformation or a collection of observations. Dot products connect geometry to similarity and weighted prediction. Linear systems, orthogonality, eigenvectors and singular-value decomposition explain fitting, projections and low-rank approximations. Rank describes independent directions of information, while conditioning describes sensitivity to perturbations. In machine learning, the same algebra appears in a dense neural layer, a covariance matrix or a factorized recommender. Understanding the dimensions and assumptions of each object is essential because a valid matrix multiplication can still represent the wrong statistical operation.

Track what each axis means before reshaping or multiplying arrays. Choose a solver suited to the matrix structure and avoid explicitly forming an inverse when solving a system is sufficient. Check rank, conditioning and the consequences of scaling features. Use decompositions to diagnose redundant variables or construct a compact representation, and verify the result through residuals or reconstruction error. A useful deliverable is a calculation whose geometry, dimensions and numerical behavior can be explained, together with checks that detect silent broadcasting and near-singular inputs.

Near-singular matrices can amplify numerical error, and finite precision matters even when an exact symbolic solution exists. Orthogonality depends on the chosen inner product, and large singular values do not automatically imply causal or predictive importance. A transpose can make dimensions compatible while changing the meaning of a calculation. Linear algebra alone does not establish statistical assumptions. Check residuals, scale and interpretation rather than accepting a result merely because the numerical library returned an array without an exception.