Proof of the variance of one-dimensional projections In Bishop's Pattern Recognition And Machine Learning book, Chapter 12,
Suppose $X$ is an uncentered data matrix and $\bar{x}=\frac{1}{m}\sum_ix_i$ is the sample mean of the columns of $X$.
For the projection onto a one-dimensional space, define an arbitrary vector $u$, which we choose to be a unit vector, so that $u^Tu=1$.
It is said in the book that the variance of the projected data is
$$u^TSu$$
where $S$ is the data covariance matrix defined by
$S = \frac{1}{N}\sum_{n=1}^{N}(x_n-\bar{x})(x_n-\bar{x})^T$
But how would I prove this?
The variance is $\frac{1}{N}\sum_{n=1}^{N}(u^Tx_n-u^T\bar{x})^2$,
But expanding this does not seem to prove it.
Can anyone help me?
 A: The mean of the data $\overline{\mathbf{x}}={1\over N}\sum_{n=1}^N \mathbf{x}_n$ is a vector averaging over all the values of $\mathbf{x}_n$ and $\mathbf{u}_1$ is a unit vector, so $\mathbf{u}_1^T \mathbf{x}_n$ in this case is just the dot product of $\mathbf{u}_1$ (the unit vector in the "principle component" direction) with the vector-valued $\mathbf{x}_n$. The left side of the equation
$$
{1\over N}\sum_{n=1}^N\{\mathbf{u}_1^T \mathbf{x}_n-\mathbf{u}_1^T \overline{\mathbf{x}}\}^2=\mathbf{u}_1^T S \mathbf{u_1}
$$
is the usual variance in the $\mathbf{u}_1$ direction, which is $\mathbf{u}_1\cdot (\mathbf{x}_n-\overline{\mathbf{x}})$ squared, so so far we just have a scalar squared within that summation. But what Bishop has done is to reverse one of the transpositions from $\mathbf{u}_1^T (\mathbf{x}_n-\overline{\mathbf{x}})$ to $(\mathbf{x}_n -\overline{\mathbf{x}})^T\mathbf{u}_1$, which can obviously be done since it's just a dot product of two vectors, and then take the constant $\mathbf{u}_1^T$ and $\mathbf{u}_1$ terms on the left and right out of the sum:
$$
\mathbf{u}_1^T
\left(
{1\over N}
\sum_{n=1}^N
\{\mathbf{x}_n-\overline{\mathbf{x}}\} 
\{\mathbf{x}_n-\overline{\mathbf{x}}\}^T
\right) \mathbf{u}_1\tag{1}
$$
and then call all the stuff in the middle $S$.
So there is not much to "prove", he's just creating a new matrix called $S$.
One thing which might be confusing is that $n$ labels the data points, not the vector indices, and $N$ is the number of data points, but $D$ is the dimension of the vectors.
To illustrate, if we let the dimension $D$ be 2, then written out in full (1) looks something like
$$
(u_1 u_2)
\times
\left(
{1\over N}
\sum_{n=1}^N
\left(
\begin{matrix}
({x_n}_1-\overline{x}_1)
({x_n}_1-\overline{x}_1)
&
({x_n}_1-\overline{x}_1)
({x_n}_2-\overline{x}_2)
\\
({x_n}_2-\overline{x}_2)
({x_n}_1-\overline{x}_1)
&
({x_n}_2-\overline{x}_2)
({x_n}_2-\overline{x}_2)
\\
\end{matrix}
\right)
\right)
\times
\left(
\begin{matrix}
u_1\\
u_2\\
\end{matrix}
\right)
$$
