My task is in numerical modeling evolution of a random quantity. For this, I need first to generate a multitude of sample sequences that correspond to a predefined correlation function. In other words, considering the process $a(\mathbf{x})$ is a function of N-dimensional vector $\mathbf{x}=(x_1,\ldots,x_N)$ and supposing its stationarity, i.e. $\langle{}a^{\ast}(\mathbf{x}')a(\mathbf{x}'')\rangle=K(\mathbf{x}''-\mathbf{x}')$, which way can I generate random samples of $a(\mathbf{x})$ that obey $K(\mathbf{x}''-\mathbf{x}')$ ?

  • $\begingroup$ Do you have any more specific requirements other than just the covariance function, e.g. that the process should be Gaussian? $\endgroup$ – Ian Sep 26 '16 at 20:15
  • $\begingroup$ It would be great if the only requirement is stationarity, however, Gaussianity can be added to first approximation. $\endgroup$ – MikhailShmulevich Sep 27 '16 at 8:35
  • $\begingroup$ I mean without something else it will be impossible to specify how to draw samples. $\endgroup$ – Ian Sep 27 '16 at 10:41
  • $\begingroup$ Ok. To be more specific, suppose that the process is Gaussian. $\endgroup$ – MikhailShmulevich Sep 27 '16 at 20:58

I'll assume you have some finite grid $\{x_1,\dots, x_n\}$ of x values on which you want to draw samples. In this case, $a$ is simply an n-dimensional multivariate gaussian with covariance matrix $C_{ij}=K(x_i-x_j)$. Many libraries (for example numpy) have builtin functionality to sample from such distributions.

However, if you need to take many samples, this can be rather slow. One trick is to do a PCA and keep only the first few components. That is, let $v_i,\lambda_i$ be the eigenvectors/eigenvalues of $C$, with $\lambda_1\ge\lambda_2\ge...$. Now fix $K$ and consider the random veector $a_K=\sum_{i\leq K} \sqrt{\lambda_i} X_iv_i$ where $X_i$ are iid unit normals. If $K=n$ (so we include all eigenvalues)then it is not hard to show that $a$ satisfies the desired covariance structure (in fact it is a gaussian process).

Often there exists $K<<n$ so that, for example $\sum_{i\leq K} \lambda_i/\sum_i\lambda_i>.95$. In this case, the truncated sum will be an excellent approximation to samples from the true distribution, and will be much faster, because each sample of $a_K$ requires $K$ independent gaussian samples, rather than the $n$ correlated samples required by the naive method.

Note this can also be done in the continuous case, and is called a Karhonen-Loeve decomosition.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.