How does a Gaussian Process define a probability distribution in the functions space?

Question

I am studying Gaussian Process Regression. I will post a text from the book Gaussian Process for Machine Learning, by C. E. Rasmussen & C. K. I. Williams:

We first consider a simple 1-d regression problem, mapping from an input x to an output f (x). In Figure 1.1 (a) we show a number of sample functions drawn at random from the prior distribution over functions specified by a particular Gaussian process which favors smooth functions. This prior is taken to represent our prior beliefs over the kinds of functions we expect to observe, before seeing any data.

The definition of a Gaussian process that I know of is given by:

Definition (Gaussian process) A random process $\left(X_{t}\right)_{t \in T}$ is called a Gaussian process if, for any finite subset $T_{0} \subset T,$ the random vector $$ \left(X_{t}\right)_{t \in T_{0}} $$ has normal distribution. Equivalently, $\left(X_{t}\right)_{t \in T}$ is Gaussian if every finite linear combination $$ \sum_{t \in T_{0}} a_{t} X_{t} $$ is a normal random variable.

I would like to better understand the connection of this definition with the possibility of defining a distribution in the functions space, because to draw functions according to a distribution, I would have to define sigma algebras in the function space and a measure of probability in this sigma algebra. Or perhaps there is another meaning when we say "sample functions drawn at random from the prior distribution over functions specified by a particular Gaussian process".

Does anyone help to build this bridge?

Answer 1

The fact that a stochastic process defines a function-valued random variable is simple enough. Say you have $X : \Omega \times T \to \mathbb{R}$. Then you can consider the mapping $\omega \mapsto (t \mapsto X(\omega,t))$ to be a function-valued random variable: for each fixed $\omega$ you have a function that maps $T$ into $\mathbb{R}$. In shorthand you would call this function $X(\omega,\cdot)$.

It's the other matters, about what the $\sigma$-algebra on the function space is and what the probability measure is, that are more technical. Rather than going into all these details here, I will simply say that:

• the usual $\sigma$-algebra on function spaces is the one generated by the so-called cylindric sets, which take the form $\{ \omega : X(\omega,t_0) \in I \}$ where $t_0$ is a fixed element of $T$ and $I$ is an interval.
• the probability measure on the cylindric sets can be shown to exist by using the so-called Kolmogorov extension theorem, once you have the finite dimensional distributions of the process.