1
$\begingroup$

In my machine learning class, we are learning about Singular Value Decomposition (SVD) on a data matrix. SVD allows us to "decompose" a $nxm$ matrix, $A$, into a product of three "simpler" matrices:

$$A = U\Sigma V^T$$ where $U$ and $V$ are unitary matrices that represent a "rotation" and $\Sigma$ is a nonnegative diagonal matrix whose entries are called the singular values and represents a "stretching." Thus, the matrix $A$ can be thought of as a linear transformation that is a composition of a rotation, stretch, and rotation.

Where I'm getting confused is how to relate the "transformation" aspect of $A$ to its physical meaning. In reality, $A$ could be a matrix of image data where each column represents a person and each row is a value for a pixel of that person's face.

Multiplying a vector, $v$, of length $m$ with $A$ would "transform" that vector (rotation, stretch, rotation) to another vector, $y$ in n-dimensional space:

$$A_{nxm}v_{mx1} = y_{_{nx1}}$$

But do the vectors $v$ and $y$ have any physical meaning like the matrix $A$ does?

$\endgroup$

1 Answer 1

1
$\begingroup$

It seems to me that your question is not related to SVD. You have a $n \times m$ matrix $A$, you take a $m \times 1$ vector $v$ and write $A v = y$ with $y$ a $n \times 1$ vector. Your question in bold is independant of any decomposition of $A$, it stands by itself.

I would answer that the physical meaning depends on the application that lead you to linear algebra. In your case, a column of $A$ is an image (a face). If the columns of $A$ are denoted $A_1, A_2, ..., A_m$, and $v = [v_1, v_2, ..., v_m]^T$, then $$ A v = \sum_{i = 1}^m v_i A_i. $$

It means that the output can be understood as an image being a weighted combination of the input images $\{A_i, i=1,2,...,m\}$. Said otherwise, $v$ contains the coordinates of the output image $y$ in the base of images defined by the columns of $A$.

$\square$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .