# Negative moments of Marchenko-Pastur law

Let $$Z \sim \mu_\lambda$$ be a the Marchenko-Pastur law with parameter $$\lambda \in (0,\infty)$$, and let $$k$$ be a negative integer

Question. Is there an analytic formula the $$k$$th moment for $$m_k(\lambda) = \mathbb E[Z^k]$$ ?

Note. I'm particularly inteteresting in $$m_{-1}(\lambda)$$ and $$m_{-2}(\lambda)$$.

## Motivation

$$m_k(\lambda)$$ is the trace of the pseudo-inverse of the $$k$$th power a Wishart random matrix (inverse covariance matrix in gaussian iid random design).

## Application: generalization error of least-squares regression

Consider a distribution $$P$$ on $$\mathbb R^d \times \mathbb R$$ defined by $$(x,y) \sim P$$ iff $$x \sim N(0,(1/d)I_d)$$ and $$y|x \sim N(w_\star^\top x,\sigma^2)$$, where $$w_\star \in \mathbb R^d$$ and $$\sigma \ge 0$$ are fixed by unknown. Thus a point drawn from $$P$$ is of the form $$(x,y)$$ where $$y=xw_\star+\eta$$, with $$\eta \sim N(0,\sigma^2)$$.

Let $$\mathcal D_n := \{(x_1,y_1),\ldots,(x_n,y_n)\} \sim P^n$$ be an iid sample from $$P$$. Consider the problem of estimating $$w_\star$$ from the data $$\mathcal D_n$$. For $$n < d$$, $$XX^\top$$ is invertible w.p $$1$$ and the least-squares solution is given by $$\hat{w} = X^\top(XX^\top)^{-1}y=P_X w_\star + X(XX^\top)^{-1}\varepsilon$$, where

• $$X$$ is the $$n \times d$$ matrix with $$i$$th row $$x_i$$,
• $$P_X := X^\top (XX^\top)^{-1} X$$ is the orthogonal projection matrix onto the row space of $$X$$,
• $$\epsilon$$ is a column vector in $$\mathbb R^n$$ with iid components from $$N(0,\sigma^2 I)$$, and
• $$y = Xw_\star+\varepsilon$$.

For any $$w \in \mathbb R^d$$, let $$f_w:\mathbb R^d \to \mathbb R$$ be the linear function $$f_w(x):=w_\star^\top x$$. Thus, the generalization error of the model $$f_{\hat{w}}$$ is given by

$$\begin{split} E_g &:= \mathbb E_{x}\mathbb E_\varepsilon[(w_\star^\top P_X x + \varepsilon^\top (XX^\top)^{-1} Xx - w_\star^\top x)^2]\\ &= (1/d)\|(I-P_X)w_\star\|^2+(\sigma^2/d)\mbox{tr}((XX^\top)^{-1}). \end{split}$$

Noise only model. For simplicity, assume $$w_\star = 0$$. Then, in the limit when $$n,d \to \infty$$ with $$n/d \to \lambda \in [0,1)$$, we have $$E_g = \sigma^2\frac{1}{d}\mbox{tr}((XX^\top)^{-1}) \to \sigma^2 m_{-1}(\lambda),\,X\text{-a.s}.$$

Thanks to the accepted answer, we conclude that $$E_g \to \dfrac{\sigma^2}{1-\lambda}$$, $$X$$-a.s.

• I just found a nice blog which computes the positive moments of the Marchenko-Pastur distribution, using that the moments of the semicircle law are the Catalan numbers: djalil.chafai.net/blog/2011/01/29/the-marchenko-pastur-law Sep 5, 2021 at 14:45
• @charmd Yes, I also stumbled on that site. Nice blog indeed. Sep 21, 2021 at 11:17

My answer will merely rely on the following reference (that I found here). It is an interesting big list of integrals, that you can read there: I.S. Gradshteyn, I.M. Ryzhik, Table of integrals, series, and products.

Denoting $$R = (\lambda^+-x)(x-\lambda^-)$$, you're interested in $$\displaystyle{\int} \frac{\sqrt{R}}{2\pi\lambda x \cdot x^m}dx$$ for $$m \ge 1$$. You can check that you have the following equalities (see 2.265):

• for $$m \ge 2$$, $$\displaystyle{\int_{\lambda^-}^{\lambda^+}} \frac{\sqrt{R}}{x^m}dx = \frac{(2m-5)(\lambda^+ + \lambda^-)}{2(m-1)\lambda^+\lambda^-} \displaystyle{\int_{\lambda^-}^{\lambda^+}} \frac{\sqrt{R}}{x^{m-1}}dx - \frac{m-4}{(m-1)\lambda^+\lambda^-}\displaystyle{\int_{\lambda^-}^{\lambda^+}} \frac{R}{x^{m-2}}dx$$

• for $$m=1$$, $$\displaystyle{\int_{\lambda^-}^{\lambda^+}} \frac{\sqrt{R}}{x}dx = 2 \pi \lambda$$.

• for $$m=0$$, $$\displaystyle{\int_{\lambda^-}^{\lambda^+}} \sqrt{R}dx = 2 \pi \lambda$$.

Thus, $$m_1(\lambda) = m_0(\lambda)=1$$, and for $$k \ge 1$$, we have the recurrence relation $$m_{-k}(\lambda) = \frac{1}{k(1-\lambda)^2} \Big((2k-3)(1+\lambda) m_{-k+1}(\lambda) - (k-3)m_{-k+2}(\lambda)\Big) \quad (\star)$$

In particular, $$m_{-1}(\lambda) = \frac{1}{(1-\lambda)^2} (-(1+\lambda) + 2) = \frac{1}{1-\lambda}$$, and $$m_{-2}(\lambda) = \frac{1}{(1-\lambda)^3}$$.

If you want additional evidence that no mistakes were made, here are the negative moments, computed either by numerical integration, or with the previous recurrence formula: they match.

Three remarks:

• Note that the formula for $$m_{-1}$$ and $$m_{-2}$$ do not generalize (no general power law expression for the moments).

• I do not know if the previous recurrence relation can lead to a straightforward formula (say, that wouldn't use a sum), but I don't believe so.

• It should also be possible to derive an exact formula for the antiderivative of $$\frac{1}{x^m} f_{MP}(x)$$, if that interests you. However, I do not believe it could be made very explicit either.

• Thanks for the detailed answer ! It also turns out I was really only interested in $m_{-k}$ for $k \in \{1,2\}$. Out of curiosity for the other cases, I opened another question here math.stackexchange.com/q/4226136/168758. Aug 16, 2021 at 22:22
• For $c \in \mathbb R$, define $M_k(\lambda; c) := \mathbb E[(Z+c)^k]$. Note that $m_k(\lambda) = M_k(\lambda;0)$. Would the computation of $M_k(\lambda;c)$ for general $c$ be much more difficult from that of $c=0$ (done in your answer) ? Sep 21, 2021 at 11:22