3
$\begingroup$

I want to generate a random vector with $\mathcal{N}(0, C)$ distribution, i.e. normal distribution with $0$ mean and given covariance matrix $C$.

$C$ is not invertible (singular). Here it's written:

The covariance matrix is allowed to be singular (in which case the corresponding distribution has no density). This case arises frequently in statistics (...)

So, how can I do it without inverting $C$?

|cite|improve this question
$\endgroup$
  • $\begingroup$ You mean not invertible. $\endgroup$ – Robert Israel Oct 27 '13 at 21:50
  • $\begingroup$ @RobertIsrael Yes, thank you. $\endgroup$ – Adam Stelmaszczyk Oct 27 '13 at 21:56
3
$\begingroup$

We are looking for a vector $BX$ with covariance matrix $C$.

$C[BX]=E(BX⋅BX^T)=E(BX⋅X^T⋅B^T)= B⋅E(XX^T)⋅B^T = BIB^T = BB^T$

So, we get matrix $B$ straight from matrix $C$, decomposing it to $BB^T$.

For this we can use LU decomposition or, when $C$ is positive definite, Cholesky decomposition.

|cite|improve this answer
$\endgroup$
1
$\begingroup$

Hint: if $B$ is a matrix and $X$ is a normal random vector with covariance matrix $I$, what is the covariance matrix of $BX$?

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Is it simply $BI$? $\endgroup$ – Adam Stelmaszczyk Oct 27 '13 at 22:34
  • $\begingroup$ No, it isn't. Use the definition of covariance matrix in terms of expected value. $\endgroup$ – Robert Israel Oct 27 '13 at 22:41
  • $\begingroup$ I don't know how to calculate covariance matrix of a... matrix, which $BX$ unfortunately is... $\endgroup$ – Adam Stelmaszczyk Oct 27 '13 at 23:48
  • $\begingroup$ $X$ is a vector. $B$ is a matrix. $BX$ is a vector (it is assumed that the sizes are appropriate so the multiplication can be done). $\endgroup$ – Robert Israel Oct 28 '13 at 0:01
  • $\begingroup$ $\mathbf{C}[BX] \triangleq E\{(\mathbf{BX}-\bar{\mathbf{BX}})(\mathbf{BX}-\bar{\mathbf{BX}})^T\}$ But I don't know what next. $\endgroup$ – Adam Stelmaszczyk Oct 28 '13 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.