I'm attempting to generalize a multivariate test of zero cross-covariance between two random varaibles to infinite dimensional Hilbert spaces and I'm looking for some advice / ideas on how to work around the lack of a standard normal distribution on infinite dimensional spaces. I have added some background information below and sketched the multivariate test.

Outer product preliminaries

Let $\mathcal{H}$ and $\mathcal{K}$ be Hilbert spaces. Throughout the following we will denote the outer product of two elements $h \in \mathcal{H}$ and $k \in \mathcal{K}$ by $h \otimes_\mathcal{H} k$ and define it as the Hilbert-Schmidt operator from $\mathcal{H}$ to $\mathcal{K}$ defined such that for $\tilde{h} \in \mathcal{H}$

$$ h \otimes_\mathcal{H} k (\tilde{h}) = \langle h , \tilde{h} \rangle_{\mathcal{H}} k $$

Note that if $V$ and $W$ are real vector spaces, this coincides with the usual matrix outer product, in the sense that for $x \in \mathbb{R}^p$ and $y \in \mathbb{R}^q$, we have

$$ x \otimes_{\mathbb{R}^p} y (\tilde{x}) = \langle x , \tilde{x} \rangle_{\mathbb{R}^p} y = x^T \tilde{x} y = y x^T \tilde{x} = (y x^T) \tilde{x} $$

so that the map $x \otimes_{\mathbb{R}^p} y$ corresponds to the matrix $y x^T$.

Mean, covariance and cross-covariance for random variables

Let $X$ and $Y$ be random variables taking values in hilbert spaces $\mathcal{H}_X$ and $\mathcal{H}_Y$ respectively. We then define the mean $m_X$ of the random variable $X$ as the unique element that satisfies

$$ \langle m_X, h \rangle = E( \langle X, h \rangle ) \quad \forall h \in \mathcal{H}_X $$

We define the covariance operator of a random variable

$$ \mathscr{C} = \textrm{Cov}(X)=E \left((X-EX) \otimes (X-EX) \right) $$

where the expectation is taken in the space of Hilbert-Schmidt operators on $\mathcal{H}_X$. Equivalently we can define it implicitly as the operator that satisfies

$$ \langle \mathscr{C}h_1, h_2 \rangle = E(\langle X, h_1 \rangle \langle X, h_2 \rangle) \quad \forall h_1, h_2 \in \mathcal{H}_X $$

Similarly, we define the cross-covariance operator of $X$ and $Y$ as

$$ \mathscr{K} = \textrm{Cov}(X, Y)=E \left((Y-EY) \otimes_{\mathcal{H}_Y} (X-EX) \right) $$

or implicitly

$$ \langle \mathscr{K} h_y, h_x \rangle_{\mathcal{H}_X} = E( \langle X, h_x \rangle_{\mathcal{H}_X} \langle Y, h_y \rangle_{\mathcal{H}_Y}) \quad \forall h_x \in \mathcal{H}_X, h_y \in \mathcal{H}_Y $$

All of the definitions above fit with the usual definitions for uni- and mulviariate real random variables.

Multivariate cross-covariance test

Consider $X$ and $Y$ to be random variables in $\mathbb{R}^{d_X}$ and $\mathbb{R}^{d_Y}$ respectively and define (under assumptions of existence of appropriate moments)

$$ \mathscr{K} = \textrm{Cov}(X,Y)=E \left((Y-EY) \otimes_{\mathbb{R}^{d_Y}} (X-EX) \right) = E((X-EX)(Y-EY)^T) $$

Assume now that $X_i$ and $Y_i$ are $n$ independent observations of $X$ and $Y$ and that we want to test if $\mathscr{K} = 0$.

A simple idea is to consider the unbiased estimate of $\mathscr{K}$:

$$ \hat{\mathscr{K}} = \frac{1}{n} \sum_{i=1}^n (X_i - \bar{X}) (Y_i-\bar{Y})^T $$

where $\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i$ and note that by the CLT, if $\mathscr{K}=0$, we would have $\sqrt{n}\hat{\mathscr{K}} \overset{\mathcal{D}}{\to} \mathcal{N}(0, \mathscr{D})$ where $\mathscr{D}$ is the covariance of $XY^T$.

Leaving out theoretical justifications, we can estimate $\mathscr{D}$ using the samples and compute $\hat{\mathscr{D}}^{-1/2}$ such that defining $$ T_n = \sqrt{n}\hat{\mathscr{D}}^{-1/2}\hat{\mathscr{K}} $$

we have $T_n \overset{\mathcal{D}}{\to} N(0, \mathscr{I})$ where $\mathscr{I}$ is the identity operator.

Thus by the continous mapping theorem, $\lVert T_n \rVert_2 \overset{\mathcal{D}}{\to} \chi^2_{d_X \cdot d_Y}$ which allows us to create tests of appropriate levels.

Infinite-dimensional cross-covariance test

Assume a setup similar to the previous section except now $X$ takes values in $\mathcal{H}_X$ and $Y$ in $\mathcal{H}_Y$ where $\mathcal{H}_X$ and $\mathcal{H}_Y$ are infinite-dimensional Hilbert spaces. We have $n$ iid. observations of $X$ and $Y$ and want to test if the cross-covariance operator is zero.

We can still estimate the operator consistently as before (under suitable moment conditions) by

$$ \hat{\mathscr{K}} = \frac{1}{n} \sum_{i=1}^n (Y_i-\bar{Y}) \otimes_{\mathcal{H}_Y} (X_i-\bar{X}) $$

however this operator will always have finite rank and thus not behave properly as a cross-covariance operator. This can be solved by regularization, thus we can have a sensible estimate of $\mathscr{K}$. I'm not quite sure that this is asymptotically normal but even if it is, we cannot "whiten" the asymptotic distribution, because the covariance operator of the asymptotic Gaussian is not invertible (since it is Hilbert-Schmidt, thus compact and therefore has finite-dimensional image).

Is there any way at all to remedy this and construct a test-statistic in a a way similar to above? Maybe theres a simpler way to test if the cross-covariance is zero but I have yet to find one that doesnt make sweeping assumptions on $X$ and $Y$. All ideas and suggestions are welcome!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.