# Signal processing, vector spaces and probability

Given a function $$f \in L^2[0, T]$$ $$f(t) = \sum_{n=0}^{N} a_n e_n(t) = \sqrt{\frac{2}{T}}\sum_{n=0}^{N} a_n \sin{\left (\frac{\pi nt}{T} \right)}$$ with the following dot product: $$\left \langle f, g \right \rangle = \int_0^Tf(t)g(t)\;dt$$ To find $$a_n$$ is easy to see that: $$a_n = \left \langle f(t), e_n(t) \right \rangle = \left \langle f(t),\sqrt{\frac{2}{T}} \sin{\left (\frac{\pi nt}{T} \right)} \right \rangle$$

Now imagine that the signal is "contaminated" with noise. $$x(t) = f(t) + \varepsilon(t)$$

$$\varepsilon(t) \sim \mathcal N(0, \sigma)$$

If you want to find the dot product we have: $$\left \langle x(t), e_n(t) \right \rangle = \left \langle f(t) + \varepsilon(t), e_n(t) \right \rangle = \left \langle f(t), e_n(t) \right \rangle + \left \langle \varepsilon(t), e_n(t) \right \rangle = a_n + \left \langle \varepsilon(t), e_n(t) \right \rangle$$ So finally we have: $$\left \langle x(t), e_n(t) \right \rangle = a_n + \left \langle \varepsilon(t), e_n(t) \right \rangle$$ $$\left \langle \varepsilon(t), e_n(t) \right \rangle = \sqrt{\frac{2}{T}}\int_0^T \varepsilon(t) \sin{\left (\frac{\pi nt}{T} \right)} dt$$ Now imagine we are trying to send a signal with the coefficients $$a_n$$, which can only take the random values of $$1$$ and $$-1$$. Both values are equiprobable $$\left(P(a_n = 1) = P(a_n = -1) = \frac{1}{2} \right)$$.

At the arrival of the contaminated signal we will decide we recieved a $$1$$ if: $$\left \langle x(t), e_n(t) \right \rangle > 0$$ and $$-1$$ if: $$\left \langle x(t), e_n(t) \right \rangle < 0$$ Then: $$P(\text{Error Reciving 1}) = P(\text{Decide -1} | \text{Sent 1}) = P(\left \langle x(t), e_n(t) \right \rangle < 0 | a_n = 1) =\\ \; \\ P(a_n + \left \langle \varepsilon(t), e_n(t) \right \rangle < 0 | a_n = 1) = P(1 + \left \langle \varepsilon(t), e_n(t) \right \rangle < 0) = P(\left \langle \varepsilon(t), e_n(t) \right \rangle < -1)\\$$ Applying the same reasoning for the error of $$-1$$, we finally get: $$P(\text{Error Reciving 1}) = P(\left \langle \varepsilon(t), e_n(t) \right \rangle < -1)$$ $$P(\text{Error Reciving -1}) = P(\left \langle \varepsilon(t), e_n(t) \right \rangle > 1)$$ The thing is, how would you find: $$P(\left \langle \varepsilon(t), e_n(t) \right \rangle < -1) \\ \; \\ P(\left \langle \varepsilon(t), e_n(t) \right \rangle > 1)$$ I've tried transforming the probability density function of $$\varepsilon(t)$$, but the transofrmation function is very complex to deal with: $$\delta(t) = g(\varepsilon(t)) = \left \langle \varepsilon(t), e_n(t) \right \rangle = \sqrt{\frac{2}{T}}\int_0^T \varepsilon(t) \sin{\left (\frac{\pi nt}{T} \right)} dt$$ $$f_\delta (\delta) = \frac{f_\varepsilon(\delta)}{|g'(\delta)|}, \; \; \; \text{where f denotes the probability density function}$$ How should I approach this? Any hint? It is possible to find an analytical form of this transformation?

• I think you want $\varepsilon(t)$ to be a Gaussian process? In which case to define it you need to specify the correlator $\mathbb{E}(\varepsilon(t_1)\varepsilon(t_2))$. Assuming this is so, I think you want the averages of $\left<f,\varepsilon\right>$ over realizations of $\varepsilon$, ie $\mathbb{E}(\left<f,\varepsilon\right>)$ and simiarly their higher moments etc. It seems like you're trying to find the 'pdf' of $\left<f,\varepsilon\right>$, but this is a much more complicated object than the pdf of a random variable. Also, where is this question from?
– Sal
Commented Apr 24 at 21:52
• @Sal So I cannot find a close form formula for the error probability? Also, answering your other question, I've made up the problem, but I do not know how to continue. Commented Apr 24 at 22:40

Let $$\begin{eqnarray*} Y & = & \langle \varepsilon, e_n \rangle\\ & = & \sqrt{\frac{2}{T}} \int_0^T \varepsilon (t) \sin \left( \frac{\pi nt}{T} \right) dt \end{eqnarray*}$$ Let's assume $$\varepsilon$$ is a mean square continuous Gaussian process, meaning for every $$t,$$ $$\varepsilon (t + h) \rightarrow \varepsilon (t)$$ in $$L^2$$ as $$h \rightarrow 0$$. Then the integral converges in $$L^2$$ so $$Y$$ is Gaussian with $$\begin{eqnarray*} E (Y) & = & \sqrt{\frac{2}{T}} \int_0^T E (\varepsilon (t)) \sin \left( \frac{\pi nt}{T} \right) dt\\ & = & 0,\\ \mathop{Var} (Y) & = & \mathop{Cov} \left( \sqrt{\frac{2}{T}} \int_0^T \varepsilon (s) \sin \left( \frac{\pi ns}{T} \right) ds, \sqrt{\frac{2}{T}} \int_0^T \varepsilon (t) \sin \left( \frac{\pi nt}{T} \right) dt \right)\\ & = & \frac{2}{T} \int_0^T \int_0^T \sin \left( \frac{\pi ns}{T} \right) \sin \left( \frac{\pi nt}{T} \right) \mathop{Cov} (\varepsilon (s), \varepsilon (t)) dsdt\\ & = & \frac{2}{T} \int_0^T \int_0^T \sin \left( \frac{\pi ns}{T} \right) \sin \left( \frac{\pi nt}{T} \right) K (s, t) dsdt, \end{eqnarray*}$$ where $$\begin{eqnarray*} K (s, t) & = & \mathop{Cov} (\varepsilon (s), \varepsilon (t)) \end{eqnarray*}$$ is the covariance kernel of the process $$\varepsilon$$.
But another assumption, formally matching your specification $$\varepsilon (t) \sim N (0, \sigma^2)$$ i.i.d, is that $$\varepsilon (t) dt = \sigma dB_t$$, where $$B_t$$ is a Brownian motion. Then $$Y$$ is Gaussian and $$\begin{eqnarray*} Y & = & \sqrt{\frac{2}{T}} \int_0^T \varepsilon (t) \sin \left( \frac{\pi nt}{T} \right) dt\\ & = & \sigma \sqrt{\frac{2}{T}} \int_0^T \sin \left( \frac{\pi nt}{T} \right) dB_t,\\ E (Y) & = & 0,\\ \mathop{Var} (Y) & = & \frac{2 \sigma^2}{T} \int_0^T \sin \left( \frac{\pi nt}{T} \right)^2 d \langle B \rangle_t\\ & = & \frac{2 \sigma^2}{T} \int_0^T \sin \left( \frac{\pi nt}{T} \right)^2 dt. \end{eqnarray*}$$
• Does this mean that $\varepsilon \sim \mathcal N(0, \sigma^2)$ and at the same time $Y \sim \mathcal N (0, \sigma^2)$? Commented May 6 at 10:50
• @ÁlvaroRodrigo The assumption $\varepsilon(t)\,dt = \sigma^2 \, dB_t$ is only formally equivalent to $\varepsilon(t) \sim N(0, \sigma^2)$ I.I.D.. Mathematically, a process with $\varepsilon(t) \sim N(0, \sigma^2)$ is too rough to be integrated. But yes, intuitively it means $\varepsilon(t)$ has the same distribution as $\langle \varepsilon, e_n \rangle$, even though $\langle \varepsilon, e_n \rangle$ can't be evaluated as an ordinary integral. Commented May 6 at 17:34
Let us start by considering the discrete time situation $$x_n = f_n+\epsilon_n,\qquad \epsilon_n \sim \mathcal N(0,\sigma),\qquad n= 1,\ldots, N.$$ Then the inner product $$I=\langle y,x\rangle = \sum_{n=1}^N y_nx_n=\sum_{n=1}^N y_n(f_n+\epsilon_n)$$ is a stochastic variable itself and since it is a sum of Gaussian variable is is also a Gaussian. In particular, we only need to compute the mean and variance to characterize it $$\mathbb E(I) = \langle y,f\rangle= \sum_n y_nf_n,\qquad \operatorname{Var}(I)=\sigma^2\langle y,y\rangle= \sigma^2\sum_n y^2_n,$$ and we conclude $$I\sim\mathcal N\left(\sum_n y_nf_n,\sigma\sqrt{\sum_n y^2_n}\right).$$ Now, we see that to have a finite continuum limit we need $$\sigma=\alpha \sqrt{\Delta t}$$ where $$\Delta t$$ is the time step. This is actually just the standard Wiener process and we could write the the inner product using stochastic integrals $$I = \int_0^T y(t)f(t)\, dt+\alpha\int_0^T y(t)\, dW(t).$$ To compute the probability of $$I>0$$, we simply use that $$I \sim \mathcal N\left(\int_0^T y(t)f(t)\, dt,\alpha \sqrt{\int_0^T y(t)^2\, dt}\right),$$ and write the probability in terms of the error function $$P(I>0)=\frac 12\left(1+\operatorname{Erf}\left[\frac{\int_0^T y(t)f(t)\, dt}{\sqrt 2 \alpha \sqrt{\int_0^T y(t)^2\, dt}}\right]\right).$$ Letting $$y(t)=\sqrt{\frac {2}{T}}\sin \frac{\pi n t}{T}$$, we find $$P(\text{Measure }a_n>0 | \text{Sent } a_n=1) = \frac 12\left(1+\operatorname{Erf}\left[\frac{a_n}{\sqrt 2 \alpha }\right]\right).$$ The other probabilities follow similarly.