1
$\begingroup$

This question has been answered before for the case where $A$ and $B$ are both square: $A^TA=B^TB$. Is $A=QB$ for some orthogonal $Q$?

I didn't really follow the algebraic explanation there but the geometric explanation in terms of hyperspheres is really nice.

Unfortunately, if $A$ is not square, then we can't use the hypersphere argument since $A$ and $B$ will map to spaces of different dimensionality. Furthermore, $P$ will not be orthogonal as it is in the above case.

So, let's suppose $A$ has dimension $k \times k$ and $B$ has dimension $d \times k$ with $k<d$. Is is still true that there exists a matrix $P$ such that $A=PB$? $P$ will need to have dimension $k \times d$.

$\endgroup$
1
  • $\begingroup$ One can find an answer here $\endgroup$
    – A.Γ.
    Sep 6, 2019 at 15:27

3 Answers 3

1
$\begingroup$

Let $B\in\mathbb R^{d\times k}$, where $k<d$. Let $M$ be a subspace of $\mathbb R^d$ such that $\text{im}B\subset M\subset\mathbb R^d$ and $\dim M = k$. Then there is a unitary operator $W : M\to\mathbb R^k$. Extend $W$ by setting $Wx =0$ for $x\in M^\perp$. Then $W : \mathbb R^d\to\mathbb R^k$ such that $W^TWy = y$ for $y\in M$. *)

Consider $C = WB\in\mathbb R^{k\times k}$. Then $C^TC = B^TW^TWB = B^TB$ and therefore $WB = C = Q(C^TC)^{1/2} = Q(B^TB)^{1/2}$. Hence, $B = W^TQ(B^TB)^{1/2}$. Setting $V = W^TQ$, you have $B = V(B^TB)^{1/2}$ with an isometry $V\in\mathbb R^{d\times k}$ (i.e., $V^TV = I_k$). Hence $$ A = U(A^TA)^{1/2} = U(B^TB)^{1/2} = UV^TB. $$


*) To see this, let $y\in M$ and $x\in \mathbb R^d$, $x = u+v$ with $u\in M$ and $v\in M^\perp$. Then $$ (W^TWy,x) = (Wy,Wx) = (Wy,Wu) = (y,u) = (y,u+v) = (y,x). $$ Thus, $W^TWy = y$.

$\endgroup$
8
  • $\begingroup$ Thanks for the reply. I understand that we can decompose B into a length preserving map $V$ and then $(B^TB)^{1/2}$ which rescales everything to get original B. Is this how I should understand this? Also, why does $V^{-1}=V^T$? $\endgroup$
    – user11128
    Sep 6, 2019 at 11:24
  • $\begingroup$ $V$ is not invertible. But $V^TV = I$. The rest of your comment I did not understand. $\endgroup$
    – amsmath
    Sep 6, 2019 at 11:25
  • $\begingroup$ I guess I don't understand why you can write $B=V(B^TB)^{1/2}$? $\endgroup$
    – user11128
    Sep 6, 2019 at 11:27
  • $\begingroup$ I edited....... $\endgroup$
    – amsmath
    Sep 6, 2019 at 11:43
  • $\begingroup$ dim $M$ = $k$ because the rank of $B$ is at most $k$? $\endgroup$
    – user11128
    Sep 6, 2019 at 11:51
1
$\begingroup$

The symmetric positive semi-definite matrix $A^TA=B^TB$ has a minimal diagonalization $V\Sigma^2V^T$, minimal in the sense of leaving out zero eigenvalues. Then $V$ is isometric, $V^TV=I_k$ and $Σ$ is diagonal of full rank $k$.

As in the classical construction of the SVD, where we aim at a factorization $A=U_AΣV^T$, $B=U_BΣV^T$, construct the matrices $U_A=AVΣ^{-1}$, $V_B=BVΣ^{-1}$, then $U_A^T U_A^{\,}=I_k$, $U_B^TU_B=I_k$ so that also these matrices are also isometric. To actually be the missing factors of the SVDs, one would have to prove that $A=[AVΣ^{-1}]ΣV^T=AVV^T$, that is, that the projector $(I-VV^T)$ has its image in the kernel of $A$, which is a standard fact for the orthogonormal eigenbasis of $A^TA$.

In conclusion, with $P=U_AU_B^T$ we get $$ PB=U_AU_B^T(U_BΣV^T)=U_AΣV^T=A. $$

$\endgroup$
9
  • $\begingroup$ Why can you conclude from $V_B\Sigma^2V_B^T = V_A\Sigma^2V_A^T$ that $V_A = V_B$? $\endgroup$
    – amsmath
    Sep 6, 2019 at 12:07
  • $\begingroup$ @amsmath : $M=A^TA=B^TB$ is the same matrix, one matrix, one diagonalization. Your objection would be valid if I had started from the SVDs of $A$ and $B$. $\endgroup$ Sep 6, 2019 at 12:12
  • $\begingroup$ Ok, then. How do you get from $B^TB = V\Sigma^2V^T$ to $B = U_B\Sigma V^T$? I know it's true, but this has to be shown (which BTW I do in my answer). $\endgroup$
    – amsmath
    Sep 6, 2019 at 12:17
  • $\begingroup$ The statement I claim you have to prove is actually nothing but the original claim (with $A = \Sigma V^T$). So, you prove the claim by using it. ;-) $\endgroup$
    – amsmath
    Sep 6, 2019 at 12:30
  • $\begingroup$ Why does the SVD of $A$ and $B$ involve the same matrix $V$ from the eigendecomposition of $A^TA=B^TB$? $\endgroup$
    – user11128
    Sep 6, 2019 at 12:52
0
$\begingroup$

Since you like the geometrical argument, I want to point out that there actually is an argument with hyperspheres and relate this geometrical viewpoint with the other answers.

Consider a matrix $B \in \mathbb{R}^{d\times k}$, which can be regarded as a a linear map $B\colon\mathbb{R}^k \to \mathbb{R}^d$. By the SVD, $B$ maps the $(k-1)$-dimensional hypersphere $S^{k-1}$ in $\mathbb{R}^k$ onto a $(n-1)$-dimensional ellipsoid in $\mathbb{R}^d$, where $n$ is the number of non-zero singular values of $B$.

  • Case 1. $k \leq d$. $B(S^{k-1})$ is an ellipsoid lying in an $k$-dimensional subspace $M$ of $\mathbb{R}^d$. If $B$ is full-rank, all singular values are non-zero and $\mathrm{im}\,B = M$. The map $B$ is an embedding of the sphere into $\mathbb{R}^d$. If $B$ doesn't have full rank, $\mathrm{im}\,B$ is a subspace of $M$. Note that in amsmath's answer the map $W$ describes how $M$ lies in $\mathbb{R}^d$.

  • Case 2. $k > d$. Also in this case there is an interpretation. The intersection of $S^{k-1}$ and $(\ker B)^\perp$ is a subsphere. This subsphere is mapped onto an ellipsoid in $\mathbb{R}^d$. So $B$ can be regarded as a kind of projection of the hypersphere onto a subsphere, and this subsphere is then mapped onto an ellipsoid.

Now let's look at your question. By the condition $A^T A = B^T B$, the ellipsoids $A(S^{k-1}) \subset \mathbb{R}^k$ and $B(S^{k-1}) \subset \mathbb{R}^d$ have axes with equal lengths. So the matrix $P$ maps the axes of the ellipsoid $B(S^{k-1})$ onto the corresponding axes of $A(S^{k-1})$. And this is exactly what the map $P$ does. Using Lutzl's notations, $P=U_A U_B^T$. Now, $U_B^T$ maps unit vectors in the directions of the axes of $B(S^{k-1})$ to the standard basis in $\mathbb{R}^k$. Consequently $U_A$ maps this basis onto the unit vectors in the directions of the axes of $A(S^{k-1})$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .