1
$\begingroup$

Suppose I have a sequence of time steps $t_1 < t_2 < t_3< \dots<t_n$. To each time step corresponds a random Gaussian variable $X_1,X_2,X_3,\dots,X_n$. Since $X = \{X_1,X_2,X_3,\dots,X_n\}$ is normally distributed, I can say that

$$ X \sim N \left( 0, \sigma^{2} \right) $$

where the distribution of $X$ has zero mean (assumption) and standard deviation $\sigma$. If I define some stochastic variable $Y_j$ such that

$$ Y_j = X_{j} - X_{j-1} $$

with $Y=\{Y_1,Y_2,\dots, Y_n\}$ for $2 \leq j \leq n$, what can I say about the distribution of $Y$? Naively, I'd think that

$$ Y \sim N \left( 0, \sigma^{2} \right) $$

which means that $Y$ is distributed the same way as $X$. My intuitive understanding is that since any $X$ is normally distributed, then any linear combination of $X$ is also normally distributed with the same mean and variance. Is this correct? If so, how can I show this rigorously?

$\endgroup$
7
  • 1
    $\begingroup$ Use the definition of $Y_j$ and the definition of variance. $\endgroup$ Commented Oct 20, 2022 at 22:09
  • 1
    $\begingroup$ Are the $X_i$ mutually independent? Otherwise it gets a lot more complicated fast. $\endgroup$
    – Matija
    Commented Oct 20, 2022 at 23:43
  • $\begingroup$ @Matija Yes. They are independent and identically distributed random variables $\endgroup$
    – kowalski
    Commented Oct 21, 2022 at 14:05
  • $\begingroup$ @RodrigodeAzevedo Can you elaborate on what you mean? $\endgroup$
    – kowalski
    Commented Oct 21, 2022 at 14:06
  • $\begingroup$ @kowalski Compute $\Bbb E \left( Y_k \right)$ and $\Bbb E \left( Y_k^2 \right)$. $\endgroup$ Commented Oct 21, 2022 at 14:40

2 Answers 2

1
$\begingroup$

Let $X\sim\mathcal N(0,\sigma^2I_n)$ be a vector of $n$ IID centered normal variables with variance $\sigma^2$ and let $Y=AX$, where $I_n$ is the identity matrix and $$ A=\begin{pmatrix} 1 & 0 & 0 & \dots & \dots & 0 & 0 & 0\\ -1 & 1 & 0 & \dots & \dots & 0 & 0 & 0\\ 0 & -1 & 1 & \dots & \dots & 0 & 0 & 0\\ 0 & 0 & -1 & \ddots & \dots & 0 & 0 & 0\\ \vdots & \vdots & \vdots & \ddots & \ddots & \vdots & \vdots & \vdots\\ 0 & 0 & 0 & \dots & \ddots & 1 & 0 & 0\\ 0 & 0 & 0 & \dots & \dots & -1 & 1 & 0\\ 0 & 0 & 0 & \dots & \dots & 0 & -1 & 1 \end{pmatrix}. $$ This gives $Y_1=X_1$ and $Y_j=X_j-X_{j-1}$ otherwise. Using integration by substitution for $X=A^{-1}Y$ (notice that $|A|=1$) and $x^{\mathrm t}$ to denote the transpose, we have \begin{aligned} \mathbb P(Y\in\mathcal E)&= \int\unicode{120793}\{Ax\in\mathcal E\}\frac{1}{\sqrt{(2\pi)^n\sigma^2}}e^{-\frac{1}{2}x^{\mathrm{t}}(\sigma^2I_n)^{-1}x}\mathrm dx\\ &=\int\unicode{120793}\{x\in\mathcal E\}\frac{|A^{-1}|}{\sqrt{(2\pi)^n\sigma^2}}e^{-\frac{1}{2}x^{\mathrm{t}}\Sigma^{-1}x}\mathrm dx, \end{aligned} where $\Sigma^{-1}=(A^{-1})^{\mathrm{t}}(\sigma^2I_n)^{-1}A^{-1}=(A(\sigma^2I_n)A^{\mathrm t})^{-1}=(\sigma^2AA^{\mathrm{t}})^{-1}$. Recall the multiplicativity of the determinant, which gives $|A^{-1}|=1$, and thereby $Y\sim\mathcal N(0,\sigma^2AA^{\mathrm{t}})$. This shows that $Y$ is a multivariate normal vector. Clearly, we want to know the covariance exactly, so $$ \Sigma=\sigma^2AA^{\mathrm{t}}=\sigma^2 \begin{pmatrix} 1 & -1 & 0 & \dots & \dots & 0 & 0 & 0\\ -1 & 2 & -1 & \dots & \dots & 0 & 0 & 0\\ 0 & -1 & 2 & \dots & \dots & 0 & 0 & 0\\ 0 & 0 & -1 & \ddots & \dots & 0 & 0 & 0\\ \vdots & \vdots & \vdots & \ddots & \ddots & \vdots & \vdots & \vdots\\ 0 & 0 & 0 & \dots & \ddots & 2 & -1 & 0\\ 0 & 0 & 0 & \dots & \dots & -1 & 2 & -1\\ 0 & 0 & 0 & \dots & \dots & 0 & -1 & 2 \end{pmatrix}. $$ Normal variables are independent iff they are uncorrelated. Thus, for example, $Y_n$ is independent of $(Y_m)_{m\le n-2}$, but clearly, the components of $Y$ are not mutually independent (since not all off-diagonal entries are $0$).

$\endgroup$
4
  • $\begingroup$ Thank you very much for your answer. I think this is what I'm looking for but I'm having some difficulties understanding it. Firstly, I understand that you used integration by substitution. I know the expression for multivariate PDF, can you elaborate what happened in the first line of the integral going to the second? $\endgroup$
    – kowalski
    Commented Oct 31, 2022 at 18:51
  • $\begingroup$ The substitution (the injective continuously differentiable map, or diffeomorphism) is $y\mapsto A^{-1}y$, with inverse $y=Ax$. The theorem tells us that when we replace $x$ by $A^{-1}y$ everywhere, we have to multiply the determinant of the Jacobian. But the Jacobian of the linear function $A^{-1}y$ is just $A^{-1}$, which explains the factor $|A^{-1}|$. When we replace $x$ in the exponent, we get $(A^{-1}x)^t(\sigma^2I)^{-1}A^{-1}x$. Then we have $(A^{-1}x)^t=x^t(A^{-1})^t$, which explains $\Sigma$. $\endgroup$
    – Matija
    Commented Oct 31, 2022 at 19:09
  • $\begingroup$ Thank you. I have had a more thorough understanding of your answer once I saw (probabilitycourse.com/chapter6/6_1_5_random_vectors.php)[this] article. One other question, how should I interpret the elements of the covariance matrix $\Sigma$? The diagonals are the variances between $X_{i},X_{i}$ or $Y_{i},Y_{i}$? Are there any $Y$ components in the $\Sigma$? $\endgroup$
    – kowalski
    Commented Oct 31, 2022 at 20:47
  • $\begingroup$ The matrix $\Sigma$ is the covariance matrix of $Y$. The diagonal entry $\Sigma_{ii}=\mathbb E[Y_i^2]$ for $i=j$ is the variance of $Y_i$ (becuase the expectation is $0$). The entry $\Sigma_{ij}=\mathbb E[Y_iY_j]$ for $i\neq j$ is the covariance of $Y_i$ and $Y_j$. If this is $0$, then the two are uncorrelated, and because this is a Gaussian distribution, also independent. $\endgroup$
    – Matija
    Commented Oct 31, 2022 at 20:58
0
$\begingroup$

Credits to @Matija for the answer, but here's my shorter take on it.

Suppose $X\sim N(0,1)$. If $Y=AX+b$, then $Y\sim N(b,A^{T}A)$

Proof:

First, denote the PDF of $X$ as $f_{X}(x)$ such that

$$ f_{X}(x) = \frac{1}{\sqrt{(2\pi)^{n}}}\text{Exp}\left(-\frac{1}{2\sigma^{2}}X^{T}X\right) $$

One can ask, given $f_{X}(x)$, what is the PDF of $Y$? Namely $f_{Y}(y)$? I claim (without proof) that if $Y=f(X)$, then injectively, $X=f^{-1}(Y)=G(Y)$ where $f^{-1}\equiv G$. Then

$$ f_{Y}(y) = f_{X}(G(y))|J| $$

where $J$ is the Jacobian, with matrix elements defined as

$$ J_{ij}=\frac{\partial{G_{i}}}{\partial Y_{j}} $$

Since $Y$ is a simple linear combination of $X$, $J$ can be easily computed:

$$ Y = Ax+b \\ X = A^{-1}(Y-b) \equiv G(Y) $$

Hence $\partial G/\partial Y = A^{-1}$ and $|J| = |A^{-1}| = |A|^{-1}$. Since we know the PDF for $X$, we can calculate $f_{X}(G(y))$:

$$ f_{X}(G(y)) = \frac{1}{\sqrt{(2\pi)^{n}}}\text{Exp}\left(-\frac{1}{2\sigma^{2}}\left(A^{-1}(Y-b)\right)^{T}\left(A^{-1}(Y-b)\right)\right) \\ =\frac{1}{\sqrt{(2\pi)^{n}}}\text{Exp}\left(-\frac{1}{2\sigma^{2}}\left((Y-b)^{T}A^{-T}\right)\left(A^{-1}(Y-b)\right)\right)\\ =\frac{1}{\sqrt{(2\pi)^{n}}}\text{Exp}\left(-\frac{1}{2\sigma^{2}}(Y-b)^{T}(A^{T}A)^{-1}(Y-b)\right)\\ =\frac{1}{\sqrt{(2\pi)^{n}}}\text{Exp}\left(-\frac{1}{2\sigma^{2}}(Y-b)^{T}(C)^{-1}(Y-b)\right) $$

where $C=A^{T}A$. Note that

$$ |C| = |A^{T}A| = |A^{T}||A|=|A|^{2} $$

hence

$$ |A| = \sqrt{|C|} \longrightarrow |A|^{-1} = \frac{1}{\sqrt{|C|}} $$

Therefore, the final PDF of $Y$ gives

$$ f_{Y}(y) = f_{X}(G(y))|J|\\ =\frac{1}{\sqrt{|C|}}f_{X}(G(y))\\ =\frac{1}{\sqrt{(2\pi)^{n}|C|}}\text{Exp}\left(-\frac{1}{2\sigma^{2}}(Y-b)^{T}(C)^{-1}(Y-b)\right) $$

But this is simply the PDF for a multivariate distribution with mean $b$ and covariance matrix $C$. Hence I can say that

$$ Y\sim N(b,C) $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .