Upperbound for Covariance Matrix Suppose $X_t \in \mathbb{R}^d$ is a vector valued time series, or in other words a vector valued stochastic process indexed by $t \in \mathbb{Z}$. Assume for the moment that $X_t$ is (weakly) stationary with $EX_t=0$, $E\|X_t\|^2<\infty$, and let
$$
C_h = E[X_0 X_h^\top]
$$
denote the autocovariance matrix at lag $h$. What I wish to show is that
$$
\|C_h\|_2 \le \|C_0\|_2,
$$
or find a counterexample to this statement. Here $\|\cdot\|_2$ is the Hilbert-Schmidt Norm:
$$\|A\|_2^2 = \sum_{i,j=1}^d a_{i,j}^2. $$
So far all I can show is the weaker statement that
$$
\|C_h\|_2 \le trace(C_0).
$$
To see this, we have by the Cauchy-Schwarz inequality for expectation and stationarity that
$$
\|C_h\|_2^2 = \sum_{i,j=1}^d (E[X_{0,i}X_{h,j}])^2 \le  \sum_{i,j=1}^d E[X_{0,i}^2]E[X_{h,j}^2] = \sum_{i,j=1}^d E[X_{0,i}^2]E[X_{0,j}^2] =[trace(C_0)]^2.
$$
Part of me believes that this bound must be sharp, i.e. there is a counter example where $\|C_h\|_2 > \|C_0\|_2$, but I really have no idea! Whatever simple examples I have tried have $\|C_h\|_2 \le \|C_0\|_2$, for instance vector autoregressive processes. Any help/advice is much appreciated.
 A: It follows from the spectral measure representation for the covariance matrix. In a nutshell, you have, $C_h=\int A(t)e^{iht}\,d\mu(t)$ where $A(t)$ are positive semidefinite and $\mu$ is some non-negative scalar measure on $[-\pi,\pi]$. Then ${\rm trace}[C_h^*C_h]=\Re\iint e^{ih(t-s)}{\rm trace}[A(t)A(s)]\,d\mu(t)d\mu(s)$. However ${\rm trace}[A(t)A(s)]={\rm trace}[A(t)^{1/2}A(s)A(t)^{1/2}]\ge 0$ for all $t,s$.
A: I think the claim is true when you have a WSS process where $E[XY^\top]$ is symmetric. I have not shown it for general $C_h$. We will show two things:
$$\frac{E[X X^\top] + E[Y Y^\top]}{2} \geq \frac{E[XY^\top] + E[YX^\top]}{2}$$
and
if $A > B$ are symmetric PSD matrices, then $||A|| > ||B||$.
Given these, your result should immediately follow (Just substitute $Y = X_h$ and use WSS to make sure variances and covariances are equal; you need symmetry in your autocorrelation function to merge the sum on the RHS into the one you want.)
For the first one, just multiply left and right by $u$, and then use the elementary Young's inequality, i.e. the fact that $E[(u^\top X)(Y^\top u)] \leq    
\frac{1}{2} (E[(u^\top X)^2] + E[(Y^\top u)^2])$
For the second, note that when you have three symmetric PSD matrices, A,B, C, and $A > B$, then we have
tr($BC$) $\leq $ tr($AC$)
which you can show by rewriting your matrices in their eigenvector form.
Thus, we have
$||B||^2$ = tr($B^\top B$) $\leq $ tr($A^\top B$) $\leq$ tr($A^\top A$) = $||A||^2$
