Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm trying to find the probability limit of

$$\hat{\beta} = \left( \sum x_i x'_i + \lambda I_k \right)^{-1} \left( \sum x_i y_i \right) $$ as $n \to \infty$, and $\lambda$ is some positive constant.

I've gotten as far as as substituting $y_i = \beta x + e_i$. (Which I know isn't very far). I'm not really sure how to progress. If anyone could provide some help that be greatly appreciated.

Thanks

share|improve this question
add comment

1 Answer

From Myung-Hoe Huh, Ingram Olkin (1984). Asymptotic aspects of ridge regression. Technical Report No. 196, Dept of Statistics, Stanford University (with some edits):

First we must assume that $\bf{X}$ is centered and scaled so that the diagonal elements of $\bf{S} = \frac{1}{n}\bf{X}'\bf{X}$ are all equal to $1$.

Assume that, as $n\rightarrow\infty$, $\bf{S}$ converges to a positive definite matrix $\bf{Q}$ that has the same eigenvalues $\ell_1,...,\ell_p$ as $\bf{S}$. As a consequence, for fixed $\lambda$ the Least Squares Estimator (LSE) $\hat{\beta}$ converges to $\beta$ in probability.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.