# Matrix as a dataset vs a transformation

In my machine learning class, we are learning about Singular Value Decomposition (SVD) on a data matrix. SVD allows us to "decompose" a $$nxm$$ matrix, $$A$$, into a product of three "simpler" matrices:

$$A = U\Sigma V^T$$ where $$U$$ and $$V$$ are unitary matrices that represent a "rotation" and $$\Sigma$$ is a nonnegative diagonal matrix whose entries are called the singular values and represents a "stretching." Thus, the matrix $$A$$ can be thought of as a linear transformation that is a composition of a rotation, stretch, and rotation.

Where I'm getting confused is how to relate the "transformation" aspect of $$A$$ to its physical meaning. In reality, $$A$$ could be a matrix of image data where each column represents a person and each row is a value for a pixel of that person's face.

Multiplying a vector, $$v$$, of length $$m$$ with $$A$$ would "transform" that vector (rotation, stretch, rotation) to another vector, $$y$$ in n-dimensional space:

$$A_{nxm}v_{mx1} = y_{_{nx1}}$$

But do the vectors $$v$$ and $$y$$ have any physical meaning like the matrix $$A$$ does?

It seems to me that your question is not related to SVD. You have a $$n \times m$$ matrix $$A$$, you take a $$m \times 1$$ vector $$v$$ and write $$A v = y$$ with $$y$$ a $$n \times 1$$ vector. Your question in bold is independant of any decomposition of $$A$$, it stands by itself.

I would answer that the physical meaning depends on the application that lead you to linear algebra. In your case, a column of $$A$$ is an image (a face). If the columns of $$A$$ are denoted $$A_1, A_2, ..., A_m$$, and $$v = [v_1, v_2, ..., v_m]^T$$, then $$A v = \sum_{i = 1}^m v_i A_i.$$

It means that the output can be understood as an image being a weighted combination of the input images $$\{A_i, i=1,2,...,m\}$$. Said otherwise, $$v$$ contains the coordinates of the output image $$y$$ in the base of images defined by the columns of $$A$$.

$$\square$$