Singular value of production of projection matrix and some other matrix

if I have a matrix $$X\in\mathbb{R}^{n\times d}$$, and some projection matrix $$P\in\mathbb{R}^{n\times n}$$ which projects things onto a $$k$$-dim subspace of the column space of $$X$$, is there some expression for the singular values of $$PX$$? Say WLOG that $$\|X\|_{op}\leq1$$. Using Weyl's inequality we get an upperbound that $$\sigma_i(PX)\leq \sigma_i(X)$$. But I suspect that this is too pessimistic since $$P$$ projects onto a subspace of $$col(X)$$. For example, if $$PX=X_k$$ being the SVD of the top $$k$$ singular values, then $$\sigma_1(PX)\leq \sigma_{k+1}(X)$$ which can be very small. Thanks for any advice!

• Please clarify the meaning of “projection” and edit your question appropriately. Do you mean an orthogonal projection or merely an idempotent matrix? Jul 31, 2023 at 8:46
• You might be interested in the work of Martin & Wang, 2009, Singular Value Assignment, which also uses projections to prove an interlacing theorem for the assignability of singular values. Jul 31, 2023 at 8:59

This is a bit of reasoning, but not a final answer.

Lemma 1: Let $$U\in\mathbb R^{n\times n}$$ be a unitary matrix and $$P\in\mathbb R^{n\times n}$$ be a projection matrix, then $$PU=UP'$$ with $$P'$$ a projection with same rank as $$P$$.

Proof : Write $$P=\sum_{i=1}^k p_i\cdot p_i^T$$ with $$\{p_i\}_{i=1}^k$$ an othonormal set of vectors. Then \begin{align*} PU&=\sum_{i=1}^k p_i\cdot (U^T p_i)^T\\ &=U\sum_{i=1}^k (U^T p_i)\cdot (U^T p_i)^T\\ &=U P' \end{align*}

It is clear from that, that we can essentially assume without loss of generality that $$X$$ is diagonal with positive values, indeed if $$X=U \Sigma V^T$$ then the question doesn't depend on $$V$$ and $$PX=UP'\Sigma V$$ where the relation between $$P$$ and $$X$$ translates equivalently to the identical relation on $$\Sigma$$ and $$P'$$. Note that the relation on $$\Sigma$$ and $$P'$$ is essentially just that $$P'=\begin{bmatrix} P''&0\\0&0 \end{bmatrix}$$ and $$\Sigma=\begin{bmatrix} \Sigma'&0\\0&0\end{bmatrix}$$ with similar shapes (the top left matrices are $$p\times p$$), therefore we can assume WLOG that $$\Sigma$$ is full rank and that $$P''$$ is any projection matrix. The question therefore becomes

If $$\Sigma$$ is a full rank diagonal matrix with positive values and $$P$$ a projection matrix, then what are the singular values of $$P\Sigma$$.

Let's have some examples to browse for intuition. Write $$\Sigma=\begin{bmatrix} \sigma_1&0&\cdots&0\\0&\sigma_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&\sigma_n \end{bmatrix}$$ with $$\sigma_1\geq \sigma_2\geq\dots\geq \sigma_n>0$$.

Example 1: Let $$P=\begin{bmatrix} I_k&0\\0&0 \end{bmatrix}$$, then the singular values of $$P\Sigma$$ are $$\sigma_1,\sigma_2,\dots, \sigma_k$$.

Example 2: Let $$P=\begin{bmatrix} 0&0\\0&I_k \end{bmatrix}$$, then the singular values of $$P\Sigma$$ are $$\sigma_{n-k+1},\sigma_{n-k+2},\dots, \sigma_n$$.

Example 3: Suppose $$n=2$$, $$k=1$$ and $$\sigma_1 > \sigma_2$$, then $$P=uu^T$$ for some unit $$u=\in\mathbb R^2$$. Observe that $$u$$ is a left singular vector with singular value $$\sqrt{u^T \Sigma^2 u}$$, indeed if $$v=(P\Sigma)^T u=\Sigma u$$ and $$P\Sigma v = P\Sigma^2 u=u \cdot (u^T \Sigma^2 u)$$.

The last example might be a bit misleading since it doesn't generalize to higher dimensions, it is not true that if $$P=\sum_{p=1}^k u_i u_i^T$$ then $$u_i$$ is a left singular vector of $$P\Sigma$$.

I can't quite finish the argument but I think this still gives a nice characterization of the extreme scenarios and a way to dive into a full characterization of all possible achievable sets of singular values for $$P\Sigma$$ and therefore for $$PX$$.