Generate sample of a random process with given correlation function My task is in numerical modeling evolution of a random quantity.
For this, I need first to  generate a multitude of sample sequences that correspond to a predefined correlation function. In other words, considering  the process $a(\mathbf{x})$  is a function of N-dimensional vector $\mathbf{x}=(x_1,\ldots,x_N)$ and supposing its stationarity, i.e. $\langle{}a^{\ast}(\mathbf{x}')a(\mathbf{x}'')\rangle=K(\mathbf{x}''-\mathbf{x}')$, which way can I generate random samples of $a(\mathbf{x})$ that obey $K(\mathbf{x}''-\mathbf{x}')$ ?
 A: I'll assume you have some finite grid  $\{x_1,\dots, x_n\}$ of x values on which you want to draw samples. In this case, $a$ is simply an n-dimensional multivariate gaussian with covariance matrix $C_{ij}=K(x_i-x_j)$. Many libraries (for example numpy) have builtin functionality to sample from such distributions. 
However, if you need to take many samples, this can be rather slow. One trick is to do a PCA and keep only the first few components. That is, let $v_i,\lambda_i$ be the eigenvectors/eigenvalues of $C$, with $\lambda_1\ge\lambda_2\ge...$. Now fix $K$ and consider the random veector  $a_K=\sum_{i\leq K} \sqrt{\lambda_i} X_iv_i$ where $X_i$ are iid unit normals. If $K=n$ (so we include all eigenvalues)then it is not hard to show that $a$ satisfies the desired covariance structure (in fact it is a gaussian process). 
Often there exists $K<<n$ so that, for example $\sum_{i\leq K} \lambda_i/\sum_i\lambda_i>.95$. In this case, the truncated sum will be an excellent approximation to samples from the true distribution, and will be much faster, because each sample of $a_K$ requires $K$ independent gaussian samples, rather than the $n$ correlated samples required by the naive method.
Note this can also be done in the continuous case, and is called a Karhonen-Loeve decomosition. 
