# If $A^TA = B^TB$ where $B$ is non-square, does $\exists$ matrix $P$ such that $A=PB$?

This question has been answered before for the case where $$A$$ and $$B$$ are both square: $A^TA=B^TB$. Is $A=QB$ for some orthogonal $Q$?

I didn't really follow the algebraic explanation there but the geometric explanation in terms of hyperspheres is really nice.

Unfortunately, if $$A$$ is not square, then we can't use the hypersphere argument since $$A$$ and $$B$$ will map to spaces of different dimensionality. Furthermore, $$P$$ will not be orthogonal as it is in the above case.

So, let's suppose $$A$$ has dimension $$k \times k$$ and $$B$$ has dimension $$d \times k$$ with $$k. Is is still true that there exists a matrix $$P$$ such that $$A=PB$$? $$P$$ will need to have dimension $$k \times d$$.

• One can find an answer here
– A.Γ.
Sep 6, 2019 at 15:27

Let $$B\in\mathbb R^{d\times k}$$, where $$k. Let $$M$$ be a subspace of $$\mathbb R^d$$ such that $$\text{im}B\subset M\subset\mathbb R^d$$ and $$\dim M = k$$. Then there is a unitary operator $$W : M\to\mathbb R^k$$. Extend $$W$$ by setting $$Wx =0$$ for $$x\in M^\perp$$. Then $$W : \mathbb R^d\to\mathbb R^k$$ such that $$W^TWy = y$$ for $$y\in M$$. *)

Consider $$C = WB\in\mathbb R^{k\times k}$$. Then $$C^TC = B^TW^TWB = B^TB$$ and therefore $$WB = C = Q(C^TC)^{1/2} = Q(B^TB)^{1/2}$$. Hence, $$B = W^TQ(B^TB)^{1/2}$$. Setting $$V = W^TQ$$, you have $$B = V(B^TB)^{1/2}$$ with an isometry $$V\in\mathbb R^{d\times k}$$ (i.e., $$V^TV = I_k$$). Hence $$A = U(A^TA)^{1/2} = U(B^TB)^{1/2} = UV^TB.$$

*) To see this, let $$y\in M$$ and $$x\in \mathbb R^d$$, $$x = u+v$$ with $$u\in M$$ and $$v\in M^\perp$$. Then $$(W^TWy,x) = (Wy,Wx) = (Wy,Wu) = (y,u) = (y,u+v) = (y,x).$$ Thus, $$W^TWy = y$$.

• Thanks for the reply. I understand that we can decompose B into a length preserving map $V$ and then $(B^TB)^{1/2}$ which rescales everything to get original B. Is this how I should understand this? Also, why does $V^{-1}=V^T$? Sep 6, 2019 at 11:24
• $V$ is not invertible. But $V^TV = I$. The rest of your comment I did not understand. Sep 6, 2019 at 11:25
• I guess I don't understand why you can write $B=V(B^TB)^{1/2}$? Sep 6, 2019 at 11:27
• I edited....... Sep 6, 2019 at 11:43
• dim $M$ = $k$ because the rank of $B$ is at most $k$? Sep 6, 2019 at 11:51

The symmetric positive semi-definite matrix $$A^TA=B^TB$$ has a minimal diagonalization $$V\Sigma^2V^T$$, minimal in the sense of leaving out zero eigenvalues. Then $$V$$ is isometric, $$V^TV=I_k$$ and $$Σ$$ is diagonal of full rank $$k$$.

As in the classical construction of the SVD, where we aim at a factorization $$A=U_AΣV^T$$, $$B=U_BΣV^T$$, construct the matrices $$U_A=AVΣ^{-1}$$, $$V_B=BVΣ^{-1}$$, then $$U_A^T U_A^{\,}=I_k$$, $$U_B^TU_B=I_k$$ so that also these matrices are also isometric. To actually be the missing factors of the SVDs, one would have to prove that $$A=[AVΣ^{-1}]ΣV^T=AVV^T$$, that is, that the projector $$(I-VV^T)$$ has its image in the kernel of $$A$$, which is a standard fact for the orthogonormal eigenbasis of $$A^TA$$.

In conclusion, with $$P=U_AU_B^T$$ we get $$PB=U_AU_B^T(U_BΣV^T)=U_AΣV^T=A.$$

• Why can you conclude from $V_B\Sigma^2V_B^T = V_A\Sigma^2V_A^T$ that $V_A = V_B$? Sep 6, 2019 at 12:07
• @amsmath : $M=A^TA=B^TB$ is the same matrix, one matrix, one diagonalization. Your objection would be valid if I had started from the SVDs of $A$ and $B$. Sep 6, 2019 at 12:12
• Ok, then. How do you get from $B^TB = V\Sigma^2V^T$ to $B = U_B\Sigma V^T$? I know it's true, but this has to be shown (which BTW I do in my answer). Sep 6, 2019 at 12:17
• The statement I claim you have to prove is actually nothing but the original claim (with $A = \Sigma V^T$). So, you prove the claim by using it. ;-) Sep 6, 2019 at 12:30
• Why does the SVD of $A$ and $B$ involve the same matrix $V$ from the eigendecomposition of $A^TA=B^TB$? Sep 6, 2019 at 12:52

Since you like the geometrical argument, I want to point out that there actually is an argument with hyperspheres and relate this geometrical viewpoint with the other answers.

Consider a matrix $$B \in \mathbb{R}^{d\times k}$$, which can be regarded as a a linear map $$B\colon\mathbb{R}^k \to \mathbb{R}^d$$. By the SVD, $$B$$ maps the $$(k-1)$$-dimensional hypersphere $$S^{k-1}$$ in $$\mathbb{R}^k$$ onto a $$(n-1)$$-dimensional ellipsoid in $$\mathbb{R}^d$$, where $$n$$ is the number of non-zero singular values of $$B$$.

• Case 1. $$k \leq d$$. $$B(S^{k-1})$$ is an ellipsoid lying in an $$k$$-dimensional subspace $$M$$ of $$\mathbb{R}^d$$. If $$B$$ is full-rank, all singular values are non-zero and $$\mathrm{im}\,B = M$$. The map $$B$$ is an embedding of the sphere into $$\mathbb{R}^d$$. If $$B$$ doesn't have full rank, $$\mathrm{im}\,B$$ is a subspace of $$M$$. Note that in amsmath's answer the map $$W$$ describes how $$M$$ lies in $$\mathbb{R}^d$$.

• Case 2. $$k > d$$. Also in this case there is an interpretation. The intersection of $$S^{k-1}$$ and $$(\ker B)^\perp$$ is a subsphere. This subsphere is mapped onto an ellipsoid in $$\mathbb{R}^d$$. So $$B$$ can be regarded as a kind of projection of the hypersphere onto a subsphere, and this subsphere is then mapped onto an ellipsoid.

Now let's look at your question. By the condition $$A^T A = B^T B$$, the ellipsoids $$A(S^{k-1}) \subset \mathbb{R}^k$$ and $$B(S^{k-1}) \subset \mathbb{R}^d$$ have axes with equal lengths. So the matrix $$P$$ maps the axes of the ellipsoid $$B(S^{k-1})$$ onto the corresponding axes of $$A(S^{k-1})$$. And this is exactly what the map $$P$$ does. Using Lutzl's notations, $$P=U_A U_B^T$$. Now, $$U_B^T$$ maps unit vectors in the directions of the axes of $$B(S^{k-1})$$ to the standard basis in $$\mathbb{R}^k$$. Consequently $$U_A$$ maps this basis onto the unit vectors in the directions of the axes of $$A(S^{k-1})$$.