# Reference request: Convergence of singular values of tall random matrix

Let $$X_n\in\mathbb{R}^{n\times m}$$ be a matrix whose entries are i.i.d. random variables with zero mean and variance $$\sigma^2$$. Let $$m$$ be a fixed integer and $$\|\cdot\|$$ denote the 2-norm of a matrix. I would like to prove that $$\left\|\frac{1}{n} X_n^\top X_n -\sigma^2 I\right\| \overset{P}{\to} 0 \ \ \ \text{ as }\ \ \ n\to\infty$$ where $$\overset{P}{\to} 0$$ denotes convergence in probability.

I believe that this should be a well-known fact, which should follow from the fact that high-dimensional random vectors are almost orthogonal as their dimension increases (see e.g. here). However, I couldn't find a reference with a rigorous proof. Hence, I would really appreciate any comment with pointers to the literature. Thanks!

In fact, since we are already assuming i.i.d. sequence, it is a direct consequence of the famous Strong (Weak) law of large numbers. (We don't need any advanced theorems other than SLLN.) The proof goes like this. Let $$X_{k,j}$$ $$(k=1,2,\ldots, n$$ ,$$j=1,2,\ldots m)$$ denote $$(k,j)$$-th entry of $$X_n$$. Observe that $$(i,j)$$-th entry of the matrix $$\frac{1}{n}X_n^\top X_n$$ is given by $$\frac{1}{n}\sum_{k=1}^n X_{ki}X_{kj}.$$ Note that for each $$(i,j)$$, $$(X_{ki}X_{kj})_{k\in\Bbb N}$$ is an i.i.d. sequence of random variables. Thus by Strong law of large numbers, we have $$\frac{1}{n}\sum_{k=1}^n X_{ki}X_{kj}\to_{\text{a.s.}} E[X_{1i}X_{1j}]=\begin{cases}\sigma^2,\quad i=j\\0,\quad i\ne j\end{cases}.$$ This implies that each entry of $$m\times m$$ matrix $$M_n:=\frac{1}{n}X_n^\top X_n-\sigma^2I$$ converges to $$0$$ almost surely. Now, since $$\|M_n\|_2^2= \sum_{i,j=1}^m |M_{i,j}|^2,$$ it follows that $$\lim_{n\to\infty}\|M_n\|_2=0$$ almost surely. It follows that $$\|\frac{1}{n}X_n^\top X_n-\sigma^2I\|_2\to_p 0,$$ since almost sure convergence implies convergence in probability.