Let $USV^T$ is a singular value decomposition of matrix $A$. In the textbook "Linear Algebra and Its Applications" by D. C. Lay et. al., where SVD is introduced, it says that "the columns of $U$ in such a decomposition are called left singular vectors of $A$, and the columns of $V$ are called right singular vectors of $A$." But it does not make any connections with the eigenvectors of $A^T\!A$. It also says that "the matrices $U$ and $V$ are not uniquely determined by $A$.
But in this web page it says that "the eigenvectors of $A^T\!A$ make up the columns of $V$, the eigenvectors of $AA^T$ make up the columns of $U$."
I'm confused about the relationship between left and right singular vectors (that is columns of $U$ and $V$) and the eigenvectors of $A^T\!A$ and $AA^T$. Any clarification is appreciated.
$\endgroup$ 11 Answer
$\begingroup$Let $A=UDV^*$. Then $$A^*A=VDU^*UDV^*=VD^2V^*\implies A^*AV=VD^2$$$$AA^*=UDV^*VDU^*=UD^2U^*\implies AA^*U=UD^2$$When a matrix is right-multiplied by a diagonal matrix, each column is multiplied by the diagonal term: $$[\mathbf{b}_1,\ldots,\mathbf{b}_n]\begin{pmatrix}\sigma^2_1&\\&\ddots\\&&\sigma_n^2\end{pmatrix}=[\sigma^2_1\mathbf{b}_1,\ldots,\sigma^2_n\mathbf{b}_n]$$
Hence the right-hand expressions are exactly the eigenvalue equations, $$A^*A\mathbf{v}_i=\sigma_i^2\mathbf{v}_i,\qquad {AA}^*\mathbf{u}_i=\sigma_i^2\mathbf{u}_i$$
$\endgroup$ 2
