Principal Component Analysis

PCA centers a numeric representation and finds directions that successively maximize projected variance subject to orthogonality. It can be computed through singular-value decomposition, yielding component directions, scores and explained-variance quantities. Keeping fewer components produces a low-rank representation and a corresponding reconstruction. Feature scale determines which variables contribute most strongly; centering does not automatically standardize them. PCA does not use target labels and therefore optimizes a different objective from supervised feature selection. Component signs can be reversed without changing the represented subspace, and correlated or similarly strong directions can complicate interpretation of individual components.

Decide whether standardization matches the meaning of features and fit all preprocessing and the PCA projection on training data. Choose component count using reconstruction, variance and downstream validation rather than a universal cutoff. Inspect loadings and representative reconstructions, and measure behavior across relevant data slices. Save centering, scaling and components for consistent inference. The result should explain what variation the reduced representation retains and loses, including whether the compression improves a downstream model or simply makes the dataset easier to visualize.

PCA captures linear variance, which can be dominated by scale, noise or nuisance variation. Low-variance directions can contain important target information. Components do not automatically correspond to interpretable or causal factors. Fitting on evaluation data leaks distributional information, and a two-dimensional plot can obscure structure lost in projection. PCA differs from nonlinear embedding methods and supervised selection. Evaluate reconstruction and task consequences, and avoid treating an explained-variance percentage as a complete measure of usefulness.