# How to derive A in this expression?

How can I derive A in this matrix represented equation(a closed form solution for ridge regression) $$\ a= (X^T X + αI)^ {−1} X^T y$$ (-1 here is inverse) and $$a = X^TA$$ I have to get $$\ α^{-1}(y-Xa) .$$ But I am not sure how can i get to it . TIA!

• Is that supposed to be an inverse symbol, a {-1} exponent? Apr 11, 2021 at 18:45
• yes, edited, thanks- it is inverse Apr 11, 2021 at 18:45
• Is "a" a matrix? It looks as if that is how you are using it; I'm just used to seeing uppercase letters for matrices and lowercase for scalars. Apr 11, 2021 at 18:47
• it is vector here Apr 11, 2021 at 18:48
• If $X$ is $n \times 1$, a column vector, then $X^T$ is $1 \times n$, a row vector, and $X^TX$ is a scalar. And $\alpha$ you say is a scalar. So doesn't that make $I$ a scalar as well? Please edit your question itself to show what are what sizes. Apr 11, 2021 at 19:07

Note that $$\ X^TX+αI$$ and its inverse are scalars. This is because $$X$$ is an $$n \times 1$$ matrix, so $$X^T$$ is $$1 \times n$$ and thus $$X^TX$$ is $$1 \times 1$$. Thus, we can move this around without worring about its order, unlike matrices.
Use the hint as the first step (after substituting the expression for $$a$$): \begin{align} \ a&= (X^T X + \alpha I)^ {−1} X^T y \\ X^TA &= (X^T X + \alpha I)^ {−1} X^T y \\ (X^T X + \alpha I)X^TA &= X^T y \\ (\alpha^{-1}X^T X + I)X^TA &= \alpha^{-1}X^T y \\ \alpha^{-1}X^T X X^T A + X^T A&= \alpha^{-1}X^T y \\ X^T A&= \alpha^{-1}X^T y - \alpha^{-1}X^T X X^T A \\ (X^T)^{-1}X^T A&= \alpha^{-1}(X^T)^{-1}X^T y - \alpha^{-1}(X^T)^{-1}X^T X X^T A \\ A&= \alpha^{-1} y - \alpha^{-1} X X^T A \\ A&= \alpha^{-1} (y - X X^T A) \\ A&= \alpha^{-1} (y - X a). \end{align} Of course, this is only valid if
• $$X^T X + \alpha I \neq 0$$, and
• $$X^T$$ is invertible, which happens if and only if $$X$$ is invertible i.e. non-singular.