# Can we anything say about $y^T y$ if we know $X^T X$ and $X^T y$

Let $$y \in \mathbb{R}^n$$ and $$X \in \mathbb{R}^{n \times p}$$, where $$n > p.$$

We don't know the matrix X, but assume we do know $$X^T X$$, and make any necessary assumptions about its rank. Assume we also know the value of $$X^T y$$.

Is there anything we can say about the value of $$y^T y$$ ?

• If the kernel of $X^T$ is non-zero, then for any $z$ in the kernel, i.e. $X^Tz=0$ the quantity $X^T(y+tz)=X^Ty$ is the same for any $t$, and nothing can be said about the norm of $y+tz$. – A.Γ. Aug 14 at 15:17

## 2 Answers

If $$\mathrm{rank}(X)=p$$, then $$P:=X(X^TX)^{-1}X^T$$ is an orthogonal projection onto the column-space of $$X$$ and $$y^Ty=\|y\|_2^2=\|Py\|_2^2+\|(I-P)y\|_2^2\geq\|Py\|_2^2=y^TPy=y^TX(X^TX)^{-1}X^Ty,$$ which is computable given that $$X^TX$$ and $$X^Ty$$ are known.

A hint could be that, we can (almost) calculate the singular value decomposition of $$X = (V\Sigma U^T)$$, since

$$(V\Sigma U^T)^T(V\Sigma U^T) = U^T\Sigma^T \underset{=I}{\underbrace{V^T V}} \Sigma U = U\Sigma^2U^T$$

Now how do singular values affect the norm?

• Hey, thanks for the response. To clarify, I also am supposing I don't know the matrix $X$. I've edited the question. – rockstar richard Aug 14 at 15:20
• Yeah, I know, but you can figure out the singular values of $X$ or $X^T$ given $X^TX$ – mathreadler Aug 14 at 15:21
• @rockstarrichard You cannot say much even if you know $X$. The case $n>p$ assure that the kernel of $X^T$ is non-trivial. Take, for example, $X=[1\ 1]^T$. Knowing $y_1+y_2=c$ does not say anything about the norm of $y$, you get the whole line of solutions. You can only get the lower bound on the $y^Ty$. – A.Γ. Aug 14 at 15:30