1
$\begingroup$

Problem

I have a matrix $$X = diag(x_1, x_2, x_3), \quad x_i \sim N \left( 0, 1 \right) \quad i = \overline{1, 3}$$ and $x_i$ are independent.

SVD is used to find singular vectors: $$X = U \cdot \Sigma \cdot V^T,$$ where $V$ and $U$ are orthogonal matrices and $\Sigma$ is diagonal matrix, such that $$\Sigma = diag\left(\sigma_1, \sigma_2, \sigma_3\right), \, \sigma_1 \geq \sigma_2 \geq \sigma_3.$$

I need to find distribution of matrix $V \cdot U^T$. But first I need to find distribution of left and right singular vectors.

In question Singular vector of random Gaussian matrix on this forum that singular vectors have uniform distribution on the sphere of radius $1$. But I don't understand how to prove it.

My solution (probably wrong)

I tried to show that distribution of $X$ is invariant with respect to rotations. To do this we need an orthogonal matrix $A \, : \, A^T = A^{-1}$ and form new matrix $$X' = A \cdot X,$$ so that distribution of $X$ and $X'$ would be the same. Though, I've found out that they don't have the same distribution. $$x_{ij}' = \sum \limits_{s = 1}^s a_{is} \cdot x_{sj} = a_{ij} \cdot x_{jj}$$ as the matrix $X$ is diagonal. Then the mathematical expectation $$\mathbb{E} x_{ij}' = a_{ij} \cdot \mathbb{E} x_{jj} = a_{ij} \cdot \mathbb{E} x_j =0$$ as $\mathbb{E} x_i = 0 \, i = \overline{1, 3}$. But the variance of $x_{ij}'$ doesn't equal 1: $$Dx_{ij}' = a_{ij}^2 \cdot Dx_{jj} = a_{ij}^2 \cdot Dx_j = a_{ij}^2 \leq 1.$$ It can be equal to $1$, but not always.

Questions

What is the right way to prove the statement, that singular vectors have uniform distribution on the sphere of radius $1$?

Will the distribution of singular vectors change if diagonal matrix elements $x_i$ has normal distribution with expectation $a$ and variance $\sigma^2 \, : \, x_i \sim N \left( a, \sigma^2 \right) $? Am I right that then sphere would have radius $a$? What if matrix $X$ isn't diagonal?

$\endgroup$
  • $\begingroup$ The vector $x = (x_1, x_2, x_3)$ follows the multivariate normal distribution $N(\mu, \Sigma)$ with $\mu = 0$ and $\Sigma = I_3$, the $3 \times 3$ identity matrix. If $y = Ax$, then $y$ follows the multivariate normal distribution $N(A\mu, A \Sigma A^T)$ (does that look familiar)? The density of a multivariate normal variable $x \sim N(\mu, \Sigma)$ is proportional to $\exp\left[ -\frac{1}{2} (x - \mu)^T \Sigma^{-1} (x - \mu) \right]$. What is the density of $y$? That shows that $X$ is invariant under rotations $\endgroup$ – Jon Warneke Mar 16 '18 at 21:00
  • $\begingroup$ @JonWarneke Thanks, it helped to understand diagonal matrix case. If the vector $x = \left( x_1, x_2, x_3 \right)$ follows the multivariate standard normal distribution $N\left( 0, I_3 \right)$ then density of $y = A \cdot x$ is the same as density of $x$. If normal distribution is not standard, then the density of $y$ is proportional to $exp \left\{ -\frac{1}{2} \left( y -A \mu \right)^T \left( A\Sigma A^T\right)^{-1} \left( y - A\mu \right) \right\}$. Distributions of $x$ and $y$ are not the same. What to do in this case? $\endgroup$ – helga-helga Mar 17 '18 at 12:21
1
$\begingroup$

First, let's write down the probability density function (PDF) of the matrix $X$; this is just the joint density of $x_1, x_2, x_3$. Since those are independent normals, we have $$ \prod_{i=1}^3 \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2} x_i^2} \, dx_i = \frac{1}{(\sqrt{2\pi})^3} e^{-\frac{1}{2} (x_1^2+x_2^2 + x_3^2)} \, dx_1 dx_2 dx_3 = \frac{1}{(\sqrt{2\pi})^3} e^{-\frac{1}{2} \operatorname{tr}(X X^T)} \, dX. $$ Now, the singular value decomposition $X = U \Sigma V^T$ is actually a change of variables (up to ordering the singular values and signs of rows/columns of $U, V^T$, but let's gloss over that) $X \to (U, \Sigma, V^T)$. In other words, you can go from the probability density of the variable $X$ to the probability density of the three variables $U, \Sigma, V^T$. To do so, substitute $X = U \Sigma V^T$ into the PDF of $X$ to obtain $$ \frac{1}{(\sqrt{2\pi})^3} e^{-\frac{1}{2} \operatorname{tr}(X X^T)} \, dX = \frac{1}{(\sqrt{2\pi})^3} e^{-\frac{1}{2} \operatorname{tr}(\Sigma^T \Sigma)} \, dX $$ using the cyclic property of $\operatorname{tr}$ and the fact that $U, V$ belong to the orthogonal group.

Recall that the differentials $dX$ and $dU \, d\Sigma \, dV$ are related by the determinant of the Jacobian matrix for the change of variables; according to this, we have $$ dX = \prod_{1 \leq i < j \leq 3} (\sigma_i^2 - \sigma_j^2) \, d\Sigma \, dU \, dV, $$ so substituting gives $$ \frac{1}{(\sqrt{2\pi})^3} e^{-\frac{1}{2} \operatorname{tr}(X X^T)} \, dX = \frac{1}{(\sqrt{2\pi})^3} e^{-\frac{1}{2} \operatorname{tr}(\Sigma^T \Sigma)} \prod_{1 \leq i < j \leq 3} (\sigma_i^2 - \sigma_j^2) \, d\Sigma \, dU \, dV. $$ Now, note that the last term is the product of three PDFs (up to normalization): one for $\Sigma$ (namely $e^{-\frac{1}{2} \operatorname{tr}(\Sigma^T \Sigma)} \prod_{1 \leq i < j \leq 3} (\sigma_i^2 - \sigma_j^2) \, d\Sigma$), one for $U$ (namely $1 \, dU$), and one for $V$ (namely $1 \, dV$). Hence, those random variables (matrices) are independent, and moreover, $U$ and $V$ are distributed uniformly over the orthogonal group. Since $U$ and $V$ are independent and uniformly distributed over the orthogonal group, so is $V U^T$.

$\endgroup$
  • $\begingroup$ Thank you for the answer. Is it true to equate the density of three-dimensional vector to the density of a matrix? $\endgroup$ – helga-helga Mar 21 '18 at 18:36
  • $\begingroup$ Yes — the density of a matrix is the joint density of its entries $\endgroup$ – Jon Warneke Mar 24 '18 at 15:28
  • $\begingroup$ In article that you attached to the answer there are wedge products in $dX$. I have also found the same expression with wedge products for $dX$ in the book. But you have wrote $dX$ in more simple form and without wedge products. Can you explain, why it is possible? $\endgroup$ – helga-helga Apr 1 '18 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.