Why is $E[AA^T]$ not necessarily invertible if $A$ is a high-dimensional vector? I'm reading a blog entry and there it says

We assume that the feature vector $\phi(x)$ is high-dimensional, so
  that the moment matrix $H=E[\phi(x)\phi(x)^{T}]$ cannot be assumed to
  be invertible.

I don't see why this claim makes any sense and does anyone here understand why this is the case? On a side note, I can't stand when authors don't justify these claims.
 A: If a random column vector $A\in\mathbb R^{n\times1}$ is constrained to lie within a $k$-dimensional subspace of $\mathbb R^{n\times1}$ with $k<n,$ then the expected value $\operatorname E(A A^\top)$ will have columns within that subspace, so its rank cannot exceed $k.$
A: Let $V=[v_1,\cdots,v_n]^T$ be a random vector $V\in\mathbb{R}^n$. Assume that the $(v_i)_i$ are independent and $v_i$ follows a law with mean $m_i$and standard deviation $\sigma_i$.
Consider $VV^T=[v_iv_j]_{i,j}$, a random symmetric $\geq 0$ matrix. 
Then $E(VV^T)=[E(v_iv_j)]$ is a symmetric $\geq 0$ matrix where, if $i\not=j$,
then $E(v_iv_j)=m_im_j$ and $E(v_i^2)=\sigma_i^2+m_i^2$. Remark that $M=[m_im_j]$ is a symmetric$\geq 0$ matrix.
Finally $E(VV^T)=M+diag(\sigma_i^2)$. If, for every $i$, $\sigma_i>0$ (that is $v_i$ is not constant), then $E(VV^T)$ is symmetric $>0$  and, in particular, invertible.
EDIT. There can be a problem when $n$ is large and the $\sigma_i/m_i$ are small.
For example, assume that the data $(v_i)$ are known with $7$ significant digits, $n=1000,m_i=1,\sigma_i^2=10^{-3}$.
Then the condition number of $E(VV^T)$ is $2\times 10^7$ and we can't calculate its inverse.
A: Notice that for any $u, v\in\Bbb R^n$, with $M = vv^T$ we have
$$Mu = vv^Tu = \langle v, u\rangle v$$
In other words, $Mu$ is always a multiple of $v$.
This means the image of $M$ has dimension $1$. It follows that whenever $n\geqslant 2$, we necessarily have that $vv^T$ fails to be invertible.
