Signal processing, vector spaces and probability

Question

Given a function $f \in L^2[0, T]$ $$ f(t) = \sum_{n=0}^{N} a_n e_n(t) = \sqrt{\frac{2}{T}}\sum_{n=0}^{N} a_n \sin{\left (\frac{\pi nt}{T} \right)} $$ with the following dot product: $$ \left \langle f, g \right \rangle = \int_0^Tf(t)g(t)\;dt $$ To find $a_n$ is easy to see that: $$ a_n = \left \langle f(t), e_n(t) \right \rangle = \left \langle f(t),\sqrt{\frac{2}{T}} \sin{\left (\frac{\pi nt}{T} \right)} \right \rangle $$

Now imagine that the signal is "contaminated" with noise. $$ x(t) = f(t) + \varepsilon(t) $$

$$ \varepsilon(t) \sim \mathcal N(0, \sigma) $$

If you want to find the dot product we have: $$ \left \langle x(t), e_n(t) \right \rangle = \left \langle f(t) + \varepsilon(t), e_n(t) \right \rangle = \left \langle f(t), e_n(t) \right \rangle + \left \langle \varepsilon(t), e_n(t) \right \rangle = a_n + \left \langle \varepsilon(t), e_n(t) \right \rangle $$ So finally we have: $$ \left \langle x(t), e_n(t) \right \rangle = a_n + \left \langle \varepsilon(t), e_n(t) \right \rangle $$ $$ \left \langle \varepsilon(t), e_n(t) \right \rangle = \sqrt{\frac{2}{T}}\int_0^T \varepsilon(t) \sin{\left (\frac{\pi nt}{T} \right)} dt $$ Now imagine we are trying to send a signal with the coefficients $a_n$, which can only take the random values of $1$ and $-1$. Both values are equiprobable $\left(P(a_n = 1) = P(a_n = -1) = \frac{1}{2} \right)$.

At the arrival of the contaminated signal we will decide we recieved a $1$ if: $$ \left \langle x(t), e_n(t) \right \rangle > 0 $$ and $-1$ if: $$ \left \langle x(t), e_n(t) \right \rangle < 0 $$ Then: $$ P(\text{Error Reciving 1}) = P(\text{Decide -1} | \text{Sent 1}) = P(\left \langle x(t), e_n(t) \right \rangle < 0 | a_n = 1) =\\ \; \\ P(a_n + \left \langle \varepsilon(t), e_n(t) \right \rangle < 0 | a_n = 1) = P(1 + \left \langle \varepsilon(t), e_n(t) \right \rangle < 0) = P(\left \langle \varepsilon(t), e_n(t) \right \rangle < -1)\\ $$ Applying the same reasoning for the error of $-1$, we finally get: $$ P(\text{Error Reciving 1}) = P(\left \langle \varepsilon(t), e_n(t) \right \rangle < -1) $$ $$ P(\text{Error Reciving -1}) = P(\left \langle \varepsilon(t), e_n(t) \right \rangle > 1) $$ The thing is, how would you find: $$ P(\left \langle \varepsilon(t), e_n(t) \right \rangle < -1) \\ \; \\ P(\left \langle \varepsilon(t), e_n(t) \right \rangle > 1) $$ I've tried transforming the probability density function of $\varepsilon(t)$, but the transofrmation function is very complex to deal with: $$ \delta(t) = g(\varepsilon(t)) = \left \langle \varepsilon(t), e_n(t) \right \rangle = \sqrt{\frac{2}{T}}\int_0^T \varepsilon(t) \sin{\left (\frac{\pi nt}{T} \right)} dt $$ $$ f_\delta (\delta) = \frac{f_\varepsilon(\delta)}{|g'(\delta)|}, \; \; \; \text{where $f$ denotes the probability density function} $$ How should I approach this? Any hint? It is possible to find an analytical form of this transformation?

Answer 1

Let \begin{eqnarray*} Y & = & \langle \varepsilon, e_n \rangle\\ & = & \sqrt{\frac{2}{T}} \int_0^T \varepsilon (t) \sin \left( \frac{\pi nt}{T} \right) dt \end{eqnarray*} Let's assume $\varepsilon$ is a mean square continuous Gaussian process, meaning for every $t,$ $\varepsilon (t + h) \rightarrow \varepsilon (t)$ in $L^2$ as $h \rightarrow 0$. Then the integral converges in $L^2$ so $Y$ is Gaussian with \begin{eqnarray*} E (Y) & = & \sqrt{\frac{2}{T}} \int_0^T E (\varepsilon (t)) \sin \left( \frac{\pi nt}{T} \right) dt\\ & = & 0,\\ \mathop{Var} (Y) & = & \mathop{Cov} \left( \sqrt{\frac{2}{T}} \int_0^T \varepsilon (s) \sin \left( \frac{\pi ns}{T} \right) ds, \sqrt{\frac{2}{T}} \int_0^T \varepsilon (t) \sin \left( \frac{\pi nt}{T} \right) dt \right)\\ & = & \frac{2}{T} \int_0^T \int_0^T \sin \left( \frac{\pi ns}{T} \right) \sin \left( \frac{\pi nt}{T} \right) \mathop{Cov} (\varepsilon (s), \varepsilon (t)) dsdt\\ & = & \frac{2}{T} \int_0^T \int_0^T \sin \left( \frac{\pi ns}{T} \right) \sin \left( \frac{\pi nt}{T} \right) K (s, t) dsdt, \end{eqnarray*} where \begin{eqnarray*} K (s, t) & = & \mathop{Cov} (\varepsilon (s), \varepsilon (t)) \end{eqnarray*} is the covariance kernel of the process $\varepsilon$.

But another assumption, formally matching your specification $\varepsilon (t) \sim N (0, \sigma^2)$ i.i.d, is that $\varepsilon (t) dt = \sigma dB_t$, where $B_t$ is a Brownian motion. Then $Y$ is Gaussian and \begin{eqnarray*} Y & = & \sqrt{\frac{2}{T}} \int_0^T \varepsilon (t) \sin \left( \frac{\pi nt}{T} \right) dt\\ & = & \sigma \sqrt{\frac{2}{T}} \int_0^T \sin \left( \frac{\pi nt}{T} \right) dB_t,\\ E (Y) & = & 0,\\ \mathop{Var} (Y) & = & \frac{2 \sigma^2}{T} \int_0^T \sin \left( \frac{\pi nt}{T} \right)^2 d \langle B \rangle_t\\ & = & \frac{2 \sigma^2}{T} \int_0^T \sin \left( \frac{\pi nt}{T} \right)^2 dt. \end{eqnarray*}

Answer 2

Let us start by considering the discrete time situation $$ x_n = f_n+\epsilon_n,\qquad \epsilon_n \sim \mathcal N(0,\sigma),\qquad n= 1,\ldots, N. $$ Then the inner product $$ I=\langle y,x\rangle = \sum_{n=1}^N y_nx_n=\sum_{n=1}^N y_n(f_n+\epsilon_n) $$ is a stochastic variable itself and since it is a sum of Gaussian variable is is also a Gaussian. In particular, we only need to compute the mean and variance to characterize it $$ \mathbb E(I) = \langle y,f\rangle= \sum_n y_nf_n,\qquad \operatorname{Var}(I)=\sigma^2\langle y,y\rangle= \sigma^2\sum_n y^2_n, $$ and we conclude $$ I\sim\mathcal N\left(\sum_n y_nf_n,\sigma\sqrt{\sum_n y^2_n}\right). $$ Now, we see that to have a finite continuum limit we need $\sigma=\alpha \sqrt{\Delta t}$ where $\Delta t$ is the time step. This is actually just the standard Wiener process and we could write the the inner product using stochastic integrals $$ I = \int_0^T y(t)f(t)\, dt+\alpha\int_0^T y(t)\, dW(t). $$ To compute the probability of $I>0$, we simply use that $$ I \sim \mathcal N\left(\int_0^T y(t)f(t)\, dt,\alpha \sqrt{\int_0^T y(t)^2\, dt}\right), $$ and write the probability in terms of the error function $$ P(I>0)=\frac 12\left(1+\operatorname{Erf}\left[\frac{\int_0^T y(t)f(t)\, dt}{\sqrt 2 \alpha \sqrt{\int_0^T y(t)^2\, dt}}\right]\right). $$ Letting $y(t)=\sqrt{\frac {2}{T}}\sin \frac{\pi n t}{T}$, we find $$ P(\text{Measure }a_n>0 | \text{Sent } a_n=1) = \frac 12\left(1+\operatorname{Erf}\left[\frac{a_n}{\sqrt 2 \alpha }\right]\right). $$ The other probabilities follow similarly.