Find the covariance of $x_{t} = x_{0}e^{-\alpha t} + \rho \int_{0}^{t} e^{-\alpha(t-s)}dW_{s}$ I want to find the covariance of the following SDE:
$$x_{t} = x_{0}e^{-\alpha t} + \rho \int_{0}^{t} e^{-\alpha(t-s)}dW_{s}$$
To start, I find the mean, it is simply:
$$ E[x_{t}] = E[x_{0}] e^{-\alpha t} $$
Then I need to compute:
$$ Cov(x_{t}, x_{s}) = E \left[ (x_{t} - E[x_{t}]) (x_{s} - E[x_{s}]) \right] $$
Expanding this out, gives me:
$$ Cov(x_{t}, x_{s}) = E\left[ (x_{0}e^{-\alpha t} + \rho \int_{0}^{t} e^{-\alpha(t-k)}dW_{k} - E[x_{0}] e^{-\alpha t}) (x_{0}e^{-\alpha s} + \rho \int_{0}^{s} e^{-\alpha(s-q)}dW_{q} - E[x_{0}] e^{-\alpha s})        \right] $$
$$ Cov(x_{t}, x_{s}) = E\left[ x_{0}^{2} e^{-\alpha(t+s)   } + x_{0} \rho\int_{0}^{s} e^{-\alpha (s-q)} dW_{q} - x_{0}E[x_{0}] e^{-\alpha(t+s)} + a_{0}\rho e^{-\alpha s} \int_{0}^{t} e^{-\alpha(t-k)} dW_{k} + \rho^{2} \int_{0}^{t} \int_{0}^{s} e^{-\alpha (t-k)} e^{-\alpha (s-q)} dW_{k} dW_{q} - \rho E[x_{0}] e^{-\alpha s} \int_{0}^{t} e^{-\alpha(t-k)} dW_{k} - x_{0}E[x_{0}] e^{-\alpha (t+s)} - \rho E[x_{0}] e^{-\alpha t} \int_{0}^{s} e^{-\alpha (s-q)} dW_{q} + E[x_{0}]^{2} e^{-\alpha(t+s)}  \right]$$
Since $E\left[\rho \int_{0}^{t} e^{-\alpha(t-k)}dW_{k} \right] = 0$, we have:
$$ Cov(x_{t}, x_{s}) =   \left(E[x_{0}^{2}] - E[x_{0}]^{2}\right)e^{-\alpha (t+s)} + \rho^{2} E\left[\int_{0}^{t} \int_{0}^{s} e^{-\alpha (t-k)} e^{-\alpha (s-q)} dW_{k} dW_{q} \right]$$
This is where I get stuck.

Question:
How can I compute $E\left[\int_{0}^{t} \int_{0}^{s} e^{-\alpha (t-k)} e^{-\alpha (s-q)} dW_{k} dW_{q} \right]$? 
It doesn't seem to follow the Wiener isometry rule.
Any ideas? 
 A: Itô's isometry shows that
$$\mathbb{E} \left( \left[ \int_0^t f(s) \, dW_s \right]^2 \right) = \mathbb{E} \left( \int_0^t f(s)^2 \, ds \right).$$
Using the polarization identity $$x \cdot y = \frac{1}{4} ((x+y)^2-(x-y)^2),$$ this gives
$$\mathbb{E} \left( \int_0^t f(s) \, dW_s \int_0^t g(r) \, dW_r \right) = \mathbb{E} \left( \int_0^t f(s) g(s) \, ds \right). \tag{1}$$
$(1)$ allows you to compute the expectation of the product of stochastic integrals, which appears in the computation of the covariance.
Proof of (1): $$\begin{align*} & \mathbb{E} \left[ \left( \int_0^t f(s) \, dW_s \right) \cdot \left( \int_0^t g(r) \, dW_r \right) \right]\\ &= \frac{1}{4} \left( \mathbb{E} \left[ \left( \int_0^t f(s)\, dW_s + \int_0^t g(r) \, dW_r \right)^2 \right] - \mathbb{E} \left[ \left( \int_0^t f(s) \, dW_s - \int_0^t g(r) \, dW_r \right)^2 \right] \right)  \\ &= \frac{1}{4} \left( \mathbb{E} \left[ \left( \int_0^t f(s)+g(s) dW_s \right)^2 \right] - \mathbb{E} \left[ \left( \int_0^t f(s)-g(s) \, dW_s \right)^2 \right] \right) \\ &= \frac{1}{4} \left( \mathbb{E} \left( \int_0^t (f(s)+g(s))^2  \, ds \right) - \mathbb{E}  \left( \int_0^t (f(s)-g(s))^2 \, ds \right)\right) \\ &= \mathbb{E} \left[ \left( \int_0^t f(s) \cdot g(s) \, ds \right) \right] \end{align*}$$
A: If $s \leq t,$
\begin{align*}
E\left[\int_{0}^{t} \int_{0}^{s} e^{-\alpha (t-k)} e^{-\alpha (s-q)} dW_{k} dW_{q} \right] &= E\left[\int_{0}^{t}e^{-\alpha (s-q)}dW_{q} \int_{0}^{s} e^{-\alpha (t-k)}  dW_{k}  \right] \\
&=E\left[\left(\int_{0}^{s}e^{-\alpha (s-q)}dW_{q} +\int_{s}^{t}e^{-\alpha (s-q)}dW_{q} \right) \int_{0}^{s} e^{-\alpha (t-k)}  dW_{k}  \right] \\
&=e^{-\alpha(t-s)}E\left[\left(\int_{0}^{s} e^{-\alpha (s-k)}  dW_{k} \right)^2 \right]+ e^{-\alpha(t-s)}E\left[\int_{s}^{t}e^{-\alpha (s-q)}dW_{q}\right] \\
&=e^{-\alpha(t-s)} E \int_{0}^{s} e^{- 2 \alpha (s-k)}  dk \\
&= \frac{1}{2 \alpha}(e^{- \alpha(t-s)}-e^{- \alpha (t+s)}).
\end{align*}
Thus, if $s > t,$ we get $\frac{1}{2 \alpha}(e^{- \alpha(s-t)}-e^{- \alpha (t+s)}).$ Therefore, the general result is $\frac{1}{2 \alpha}(e^{- \alpha |s-t|}-e^{- \alpha (t+s)}).$
(The fourth equality follows from Ito's Isometry and the fact that $E\int_{s}^{t}e^{-\alpha (s-q)}dW_{q} = 0$)
