Suppose that $(X_1,.....,X_n)$ follows multivariate normal with
$${E(X_i) = 0 \;,\; 1\le i \le n\; ;} $$ $${Var(X_i) = 1\;,\; 1\le i \le n\; ;} $$ $${Corr(X_i,X_j) = \rho\;;\;1\le i\;\neq j \le n\; ;}$$ for $ some\; 0\le \rho < 1$.
Show that for any ${\rho\in [0,1),(X_1,.....,X_n)}$ as above exist, but not necessarily for $\rho\in (-1,0]$.

I thought of doing it using density function of multivariate normal
That is,if density of $(X_1,.....,X_n)$ is g(x) where $x=(x_1,...x_n) \in \mathbb{R^n}$
$${{g(x)=\frac{1}{{(2\pi)^\frac n2}\sqrt{\det\Sigma}}e^{-\frac12 x^T\Sigma^{-1}x},1(x\in \mathbb{R^n})\;;}}$$
where $\Sigma = \begin{bmatrix} 1 & \rho &\cdots&\cdots &\rho \\ \rho & 1& \ddots &&\rho \\ \vdots&\ddots&\ddots&\ddots &\vdots\\ \rho&&&1&\rho\\ \rho &\cdots&&\rho&1\\ \end{bmatrix} _{n\times n}$
${\det\Sigma = (1+(n-1)\rho){(1-\rho)}^{n-1}} $
I further thought that as $\det\Sigma$ need to be $\ge 0$ we can come up with some conditions that $\rho$ need to satisfy.But I didn't get anything useful. Maybe my thought process was entirely wrong.
Anyway I would be thankful for any help

  • $\begingroup$ Perhaps stop the craziness with enlarged fonts? $\endgroup$ – Did Dec 3 '18 at 6:54
  • $\begingroup$ I wrote the density in enlarged fonts,so that it is more legible.the problem with your edit is that we have to strain our eyes to read the density especially the exponential part.I prefer not to strain thats all $\endgroup$ – Amelia Dec 3 '18 at 7:10
  • $\begingroup$ Quite bad idea. If you have doubts, look around you on the site and wonder why nobody does it. $\endgroup$ – Did Dec 3 '18 at 7:14
  • $\begingroup$ Why is what so? Why is it a terrible idea to add \LARGE and \large everywhere randomly? If you are truly interested in understanding why, perhaps follow the suggestion in my previous comment. Re the exponential part, simply use \exp(...) instead of e^{...}. $\endgroup$ – Did Dec 3 '18 at 7:17
  • $\begingroup$ Ok thanks.As I mentioned before this was my first time .So sorry for the inconvenience if any $\endgroup$ – Amelia Dec 3 '18 at 7:22

The $\Sigma$ you desire can be written as $(1-\rho)I + \rho J$ where $I$ is the identity matrix and $J$ is the matrix of all $1$s. Can you compute the eigenvalues of $\Sigma$ now, and state a condition for $\Sigma\succeq 0$?

The eigenvectors of $J$ are the all ones vector (with eigenvalue $n$) and the vectors orthogonal to the all ones vector (with eigenvalue $0$). Thus the eigenvalues of $\Sigma$ are $1-\rho + n\rho$ and $1 - \rho$. So if $\rho \in [0, 1)$, these eigenvalues are nonnegative so $\Sigma \succeq 0$. If $\rho \in (-1, 0]$, then you can find an $n$ such that $1 - \rho + n \rho < 0$ so $\Sigma \not \succeq 0$.

  • $\begingroup$ Actually, unless each Xi has unit variance, the correlation matrix isn't the same as the covariance matrix. (They are, up to a scaling, though) $\endgroup$ – Vim Dec 3 '18 at 6:07
  • $\begingroup$ I get the point that eigen values of $\Sigma\; are\; (1-\rho + n\rho)\;and \rho\; $. $\endgroup$ – Amelia Dec 3 '18 at 6:32
  • $\begingroup$ after that can u please check if my reasoning is correct $\endgroup$ – Amelia Dec 3 '18 at 6:32
  • 1
    $\begingroup$ so for $\Sigma$ to be semi positive definite all its eigen values must be $\ge$ 0.Thus $\rho\;\le\;1$ and $\rho \ge\frac{-1}{n-1}$ for any value of n.Thus if $n\to \infty$ then$\rho\ge 0$ $\endgroup$ – Amelia Dec 3 '18 at 6:35
  • $\begingroup$ @Amelia See my edit. $\endgroup$ – angryavian Dec 3 '18 at 6:36

Here's the idea: let $e$ be the n-dim standard normal, then a linear transformation of $e$, say $X=\mu+Ae$ is again multivariate normal (prove it by using, say, characteristic functions for multivariate distributions) whose covariance is clearly $AA^T$. Now what $A$ can satisfy your requirement?

To elaborate: for existence for rho in [0, 1), rewrite the required correlation matrix as $$C=\rho 11^T + (1-\rho)I$$ And therefore the required covariance matrix is just $$\Sigma=vCv^T,\quad\text{where}\,v=[\sigma(X_1)^2,\cdots \sigma(X_n)^2]^T.$$ which is positive definite and admits a non-singular Cholesky decomposition.

For $\rho\in(-1,0]$, $C$ isn't even positive semidefinite (proof easy), and therefore cannot be a correlation matrix.

Edit: sorry the last statement is wrong. Whether or not $C$ is positive semidefinite seems to depend on how large $\rho$ is. Actually, your required distribution exists if and only if the correlation matrix $C$ is semi-positive definite (in which case a Cholesky decomposition is possible). I'm talking only if because correlation matrix must be PSD.

For $\rho<0$, when does $C$ fail to be PSD? When there exists a non zero $x\in\Bbb R^n$ such that $$x^TCx = \rho (1^T x)^2 + (1-\rho)\|x\|^2 < 0$$ Or $$|1^Tx| / \|x\|> \left(\frac{1 + |\rho|}{|\rho|}\right)^{1/2}.$$ But since $x\mapsto 1^Tx$ is a bounded linear map, this cannot happend with a too small $|\rho|$ (in which case RHS would be too large). Indeed, the above inequality has solution if and only if the matrix norm (spectrual radius, namely) $\|(x\mapsto 1^Tx)\| > \left(\frac{1 + |\rho|}{|\rho|}\right)^{1/2}$.

  • $\begingroup$ so $\Sigma=AA^T$.For this to happen $\Sigma$ must be a positive definite matrix -By cholesky decomposition.Is that what u meant $\endgroup$ – Amelia Dec 3 '18 at 5:56
  • $\begingroup$ Yes please see edit. $\endgroup$ – Vim Dec 3 '18 at 5:59
  • $\begingroup$ thanks @viv.Yours was a more general proof and I appreciate your help.If u find my question acceptable can u please up vote it.I am a newbie and I lack in reputation that's why.I am sorry to bother – Amelia 1 min ago edit $\endgroup$ – Amelia Dec 3 '18 at 6:47
  • $\begingroup$ @Jean-ClaudeArbaut sorry you're right. If rho is a small negative number, then it's still positive semidefinite because $11^T$ is a bounded linear operator. $\endgroup$ – Vim Dec 3 '18 at 8:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.