5
$\begingroup$

Kind of embarrassing, but I'm completely blanking on what applying the expectation operator to a matrix means, and I can't find a simple explanation anywhere, or an example of how to carry out the computation. I'm learning about Wiener filtering, which can be used for a number of different purposes. Basically, the Wiener filter works as as mean-square-error minimization between the input signal and the "target" signal. The signal is assumed to be composed of our target and noise :

$$ x[n] = y[n] + w[n] $$

Where x is the input signal, y is the target and w is noise. All are assumed to be wide-sense stationary random processes, meaning that the mean and covariance do not change in time.

Where I'm getting hung up is that the covariance of x with respect to itself (the variance matrix) is calculated as

$$ C_{x} = E[xx^{T}] $$

but I don't remember what it means to apply the expectation operator to a matrix (in this case, xxT).

EDIT:

Perhaps, I should be more clear in that what I don't understand is how to apply the expectation operator to the individual values within the matrix. For instance, the entry in the first row, first column of $$C_{x}$$ will be $$x[0] * x[0] = x[0]^{2}$$. How does one calculate $$E[x[0]^{2}]$$, and what does that mean?

$\endgroup$
  • $\begingroup$ just apply $E[.]$ to each component of the matrix $\endgroup$ – user66081 Jan 18 '14 at 4:07
  • $\begingroup$ W"ie"ner filtering $\endgroup$ – user66081 Jan 18 '14 at 4:08
  • $\begingroup$ That's what I'm uncertain about in this case. For example (assuming n=0...N) if we look at $$C_{x}$$ at the location [0,0] we are finding the expected value of x[0]*x[0] or $$E[x[0]*x[0]] = E[x[0]^{2}]$$ I'm not sure what it means to apply the expectation operator to a number, not a random variable. $\endgroup$ – themantalope Jan 18 '14 at 4:28
5
$\begingroup$

The expected value of matrix $A$, $E(A)$, is just a matrix containing the expected values of the elements of the matrix $A$, as has already been mentioned.
Moreover, a number has no expected value, being a constant. The expression $E(XX^T)$ refers to the $X's$ viewed as random variables, and not as the specific realizations that you have available.

So the expression in the upper most left corner will be $$E(X_0X_0) = E(X_0^2)$$ but you are not supposed to insert in place of $X_0$ the actual value $x_0$ you have available as a realization of the random variable $X_0$. This is a theoretical expression.

If you want to obtain an estimate of these expected values, you have to invoke the covariance-stationarity assumption. When processes are stationary, any continuous transformation or combination of them is also stationary, and also, sample means are consistent estimators of theoretical moments.

Covariance stationarity implies $E(X_0^2) = E(X_1^2)=... =E(X_n^2)$

Define $Z_0= X_0^2$ etc. Then $E(Z_0) = E(Z_1^2)=... E(Z_n^2)$, and the $Z$'s are stationary. If they are stationary, the expression

$$\frac 1{n+1} \sum_{i=0}^{n}z_i \rightarrow_p E(Z_0)= E(X_0^2)$$

and the lowercase letters denote the actual realizations, the data sample you have available.

Moreover, covariance stationarity implies not necessarily that the covariances are constant through time, but that they only depend on the distance between two elements, and not on their exact position in the sequence (i.e. not on the time index). So a priori you assume that, for example, $E(X_0X_1) = E(X_1X_2) =...= E(X_{n-1}X_n)$. If you want to estimate this one-step covariance, by the same argument as before, you can calculate

$$\frac 1{n} \sum_{i=1}^{n}x_ix_{i-1} \rightarrow_p E(X_1X_2)$$

Note that the longer the gap between two variables, the more information from your sample you are going to lose in estimating their covariance.

Finally note that the expression $E(XX^T)$ is the (auto)covariance matrix only if the $X$'s have zero expected value.

$\endgroup$
  • $\begingroup$ thanks for the explanation. I did some more reading on the topic, and as you said I now know that the best I can do is find the estimation of that matrix. But again thanks for the explanation, it really help me improve my understanding. $\endgroup$ – themantalope Jan 20 '14 at 15:27
0
$\begingroup$

$$ E[x x^T] = E[(y+n) (y+n)^T] = E[yy^T + y n^T + n y^T + nn^T]\\ = R_{yy} + R_{yn} + R_{ny} + R_{nn} \\= R_{yy} + R_{nn} ~~\text{ if $y$ and $n$ are uncorrelated.}$$

$\endgroup$
  • $\begingroup$ I know that, what I'm uncertain about is how to compute $$E[xx^{T}]$$. How is the expected value of a number (not a random variable) computed? $\endgroup$ – themantalope Jan 18 '14 at 6:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.