The goal is to design a full, true test of independence for a sequence of multivariate observations $X_1,X_2,\dots,X_n$, that is, testing if the observations (each assuming to come from a same distribution) are jointly independently distributed. The data may be coming from dynamical systems, the variance and/or expectation may be infinite, and as result the autocorrelations may not exist. This has nothing to do with $\chi^2$ tests or testing if autocorrelations are zero.

However, I will only discuss here the most basic case. I am interested to know if the statistic of the test, in the most basic case, is asymptotically normal and does not depend on the distribution of the $X_k$'s. Assume the $X_k$'s are univariate and we want to test if $X_k$ is independent of $X_{k+1}$.

So here's the test. You compute the empirical joint probabilities $p_{\alpha,\beta}=P(X_k\leq\alpha, X_{k+1}\leq\beta)$ for various values of $0\leq\alpha,\beta\leq1$. You also compute the empirical marginal probabilities $p_\alpha=P(X_k\leq \alpha)$. The statistic of the test is simply $$D=\sigma\sum_{\alpha,\beta}|p_{\alpha,\beta}-p_\alpha p_\beta|$$

where $\sigma$ is a normalizing constant. Under which conditions is $D$ asymptotically normal under $H_0$, that is in case of true independence? How to choose $\sigma$ and how to choose the $\alpha,\beta$ to have a sound test? Are there already any test achieving the same purpose? For simplicity, we can assume that $X_k\in[0,1]$, and the number $n$ of observations is large, say $n=10^6$.


Let $N$ be the number of vectors $(\alpha,\beta)$ for which $p_{\alpha,\beta}$ is computed. In my case $N\approx 4000$, the $\alpha,\beta$ are evenly sampled in $[0,1]$, and $X_k\in [0,1]$. It makes sense to use $\sigma$ proportional to $N$. Note that the $p_{\alpha,\beta}$ are not independent. This is the price to pay for not using bins (some of which could be very sparse) together with a standard $\chi^2$ test. The theoretical distribution of $D$ can be approximated by simulation (resampling-like methods). It still should be asymptotically normal. Maybe a better statistic is

$$D^*=\sup_{\alpha,\beta}|p_{\alpha,\beta}-p_\alpha p_\beta|.$$

This makes the test more like a Kolmogorov-Smirnov one, rather than $\chi^2$-like. Also, in my case, I have the ability to produce as many samples $X_1,\cdots,X_n$ as I want, with $n$ as large as I want. One example is to start with $X_1$ a random deviate in $[0,1]$, and $X_{k+1}=bX_k - \lfloor b X_k\rfloor$, where the brackets represent the integer part function and $b>1$ is an integer. In that case, it is well known that the theoretical lag-$m$ autocorrelation in the infinite sequence $(X_k)$ is equal to $1/b^m$, and thus close to zero for large values of $b$. Also, the theoretical value of $p_\alpha$ is $p_\alpha=\alpha$. I am not interested in that case because all the theory is known, but in more complicated dynamical systems.

  • $\begingroup$ In the example cited in the 'Update' section, computational errors for $X_k$ propagate exponentially fast, but the system is ergodic, so this is not a concern. $\endgroup$ Mar 28 at 2:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.