Ridge Regression: $\hat{\beta} \rightarrow \beta$

Question

I'm trying to find the probability limit of

$$\hat{\beta} = \left( \sum x_i x'_i + \lambda I_k \right)^{-1} \left( \sum x_i y_i \right) $$ as $n \to \infty$, and $\lambda$ is some positive constant.

I've gotten as far as as substituting $y_i = \beta x + e_i$. (Which I know isn't very far). I'm not really sure how to progress. If anyone could provide some help that be greatly appreciated.

Thanks

Answer 1

From Myung-Hoe Huh, Ingram Olkin (1984). Asymptotic aspects of ridge regression. Technical Report No. 196, Dept of Statistics, Stanford University (with some edits):

First we must assume that $\bf{X}$ is centered and scaled so that the diagonal elements of $\bf{S} = \frac{1}{n}\bf{X}'\bf{X}$ are all equal to $1$.

Assume that, as $n\rightarrow\infty$, $\bf{S}$ converges to a positive definite matrix $\bf{Q}$ that has the same eigenvalues $\ell_1,...,\ell_p$ as $\bf{S}$. As a consequence, for fixed $\lambda$ the Least Squares Estimator (LSE) $\hat{\beta}$ converges to $\beta$ in probability.