Two definitions of a Stochastic Process?

Question

I have two supposedly equivalent definitions of a stochastic process.

1. A stochastic process is an indexed set of random variables. Specifically $$ y = \{y(x) \; | \; x \in \mathcal{X}\}. $$ Typically $y \in \mathbb{R}$ and $\mathcal{X} \subset \mathbb{R}^n$. If the index set is infinite, then we might say that the stochastic process is continuous.
2. A stochastic process is a probability distribution over a space of functions.

I'm trying to read through Gaussian Processes for Machine Learning by Rasmussen and Williams, and they mention that you can consider a Gaussian Process to be a distribution over a space of functions.

Question: I have the definition that a Gaussian process is a stochastic process (in the sense of definition 1.) for which every finite collection of random variables has the property that $$ (y(x_1),...,y(x_n)) \sim \mathcal{N}(\mathbf{\mu},\mathbf{\Sigma}), (\forall \; n \in \mathbb{N} \text{ and } x_1,...,x_n). $$ How can I reconcile this definition with the perspective from definition (2)? I guess that I should be thinking about $y$ as being a function so $y(\cdot)$ is more appropriate, but then is $y(\cdot)$ a collection of random functions? How does it define a distribution, and can it be written down explicitly? I hope my questions are clear, I'm new to the material with minimal background knowledge. Thanks in advance for the help!

Answer 1

In the definition you start with, $$\tag{1} (y(x_1),...,y(x_n)) \sim \mathcal{N}(\mathbf{\mu}_n,\mathbf{\Sigma}_n),\quad (\forall \; n \in \mathbb{N} \text{ and } x_1,...,x_n). $$ we need labels at the mean vector $\mu_n$ and at the covariance matrix $\Sigma_n$.

The index set of this process is the space the $x_i$ live in. It seems you mean $x_i\in\mathbb R^\color{red}{d}$ for some $\color{red}{d}$. So ${\cal X}\subset\mathbb R^d$. (We should not use $n$ twice.) Further it is clear that $y(x_i)\in\mathbb R$.

We can now look at $y$ as being a function from ${\cal X}$ to $\mathbb R$ and definition (1) says that for each $n$ and every vecor $(x_1,...,x_n)$ the random variable $(y(x_1),...,y(x_n))$ is $n$-variate Gaussian. In other words, the distribution of $(y(x_1),...,y(x_n))$ is a finite dimensional Gaussian probability measure on $\mathbb R^n$.

Note that $$ (x_1,...,x_n)\mapsto (y(x_1),...,y(x_n)) $$ is a function from $\mathbb R^n$ to $\mathbb R^n$ whilst $y$ is a function from a subset of $\mathbb R^\color{red}{d}$ to $\mathbb R$.

The not very trival but intuitively plausible Kolmogorov extension theorem says under what conditions there exists an infinite dimensional Gaussian measure on the set of functions $y:\mathbb R^\color{red}{d}\to \mathbb R$ that is compatible with the finite dimensional one above in the sense that (1) holds. Strictly speaking we encounter this theorem mostly in the context of the Wiener process. I hope you are not asking me now to prove it for (1). Getting the intuition is I think more important at that stage.