Largest eigenvalue of covariance matrix

Consider a random vector $$X\in\mathbb{R}^d$$ that is distributed according to a multivariate Gaussian $$\mathcal{N}(\mu, \Sigma)$$, and the random variable $$y = \langle a, X \rangle + e$$ for some (fixed) vector $$a\in\mathbb{R}^d$$ and random variable $$e\in\mathbb{R}$$ that is independent of $$X$$ and distributed according to a centered univariate Gaussian $$\mathcal{N}(0, \sigma_e^2)$$.

It is well-known that $$y$$ is therefore distributed according as a $$\mathcal{N}(a^T \mu, a^T \Sigma a + \sigma_e^2)$$.

I am interested in the eigenvalues of the covariance matrix of the random vector $$F = \begin{bmatrix} X \\ y \end{bmatrix} \in\mathbb{R}^{d+1}$$ which also has a multivariate Gaussian distribution, according to the above.

This covariance matrix (let's call it $$Z$$) is a block matrix, as follows: $$Z = \begin{bmatrix} \Sigma & B \\ B^T & \sigma_e^2 + a^T \Sigma a \end{bmatrix}$$ for some appropriate vector $$B$$ that is the covariance of $$X$$ and $$y$$.

If it is not possible to exactly analytically find the largest eigenvalue of $$Z$$, then I would like to arrive at a (good) upper bound of it. I suspect that it would look something like $$\lambda_\text{max}(\Sigma)$$ plus something like $$\sigma_e^2 + a^T \Sigma a$$.

• Since $d \geq 0$ and $X$ is positive semidefinite by construction, this seems like something you could use the Schur complement for... Jun 1, 2021 at 1:45
• @user594147 sure, but the eigenvalues which are the point here, do not seem to be directly derived from that.
– Jay
Jun 1, 2021 at 23:06

Firstly, $$B=\Sigma a$$, because $$\sigma$$ is a symmetric positive definite matrix, there are an orthonormal matrix $$U \in \mathbb{R}^{d\times d}$$ and a positive diagonal matrix $$\Lambda \in \mathbb{R}^{d\times d}$$such that $$U^T \Lambda U= \Sigma$$, hence \begin{align}\begin{bmatrix} \Sigma & B \\ B^T & \sigma_e^2 + a^T \Sigma a \end{bmatrix} &= \underbrace{\begin{bmatrix} U^T & 0 \\ 0 & 1 \end{bmatrix}}_{=:C} \begin{bmatrix} \Lambda & \Lambda Ua \\ a^TU^T\Lambda & \sigma_e^2 + a^T U^T \Lambda Ua \end{bmatrix} \underbrace{\begin{bmatrix} U & 0 \\ 0 & 1 \end{bmatrix}}_{=C^T} \\ &= C^T \underbrace{\begin{bmatrix} \sqrt{\Lambda} & 0\\ a^TU^T\sqrt{\Lambda} & \sigma_e \end{bmatrix}}_{=D} \begin{bmatrix} \sqrt{\Lambda} & \sqrt{\Lambda} Ua \\ 0 & \sigma_e \end{bmatrix} C\\ &=C^TDD^TC \end{align} Because $$C$$ is also an orthonormal matrix, $$\|Z\|=\| DD^T\|= \| D\|^2$$, where $$\| \cdot \|$$ is the usual matrix norm. Besides, $$D=\left\|\begin{bmatrix} \sqrt{\Lambda} & \sqrt{\Lambda} Ua \\ 0 & \sigma_e \end{bmatrix} \right\| \le \left\|\begin{bmatrix} \sqrt{\Lambda} & 0 \\ 0 & \sigma_e \end{bmatrix} \right\|+\left\|\begin{bmatrix} 0 & \sqrt{\Lambda} Ua \\ 0 & 0 \end{bmatrix} \right\| =\max( \| \Lambda \|,\sigma_e)+\sqrt{a^TU^T\Lambda U a}=\sqrt{\max(|\Sigma|,\sigma_e^2)}+\sqrt{ a^T\Sigma a}$$
$$\|Z\| \le \left(\sqrt{\max(|\Sigma|,\sigma_e^2)}+\sqrt{ a^T\Sigma a}\right)^2$$