The early History of the Singular Value Decomposition (1993) [pdf]

For the curious, eigenvalues only exist for square matrices. Singular values are like generalized eigenvalues.

Singular values are like the fundamental frequencies of your matrix. You know how you can define any color with RGB? In a (pretty handwavy) way, singular values are like RGB color codes for us math guys.

Optimizers like Muon and Adam play around with weights' first, or second order singular values to train models.

The SVD seems to come up everywhere in my work in computer vision. I find myself continuously using the various C++/Eigen SVD implementations. Actually I should speak in the past tense. Claude and Codex are now generating all my code for me now, and I see them spitting out SVD code frequently -- often for very special cases. SVD truly is an amazing tool.