I'm currently doing some work with Particle Filters, a sampling-based method for computing expectations of functions with respect to dynamic (ie: time-variant) random variables. For example, consider a Markov chain where you only get to observe $Y_t \sim P(Y_t|x_t)$ for unknown variables $X_t$. You would like to calculate $\mathbb{E}[f(X_t)|y_{1:t}]$ for all $t=1\ldots T$.

To the point, Particle Filters traditionally involve a "resampling" step, and I'm curious whether or not it will result in an unbiased estimate of $f$.

Suppose you have 3 random variables $X,Y,Z$. $Z$ is conditionally independent of $X$ given $Y$. You would like samples from $P(Y|Z)$ given a finite set of $N$ samples $x^{i}$ from $P(X)$ such that the estimate $\frac{1}{N}\sum_{i} w_{i}f(y^{i})$ is unbiased of $\mathbb{E}[f(Y)|Z=z]$ for some weights $w_i$.

I begin by sampling $y_{tmp}^{i} \sim P(y^{i} | x^i)$. I then assign each sample a weight $b_{i} = P(z|y^{i}_{tmp})$ for an observed realization for $z$. I "resample" $y^{i}$ from $y_{tmp}^i$ $N$ times according to a multinomial distribution with weights $b_i$, then assign each $y^{i}$ weights $w_{i} = \frac{1}{N}$.

Do the weighted mean of the resulting values $y^{i}$ give an unbiased estimate for $\mathbb{E}[f(Y)|Z=z]$ for some reasonable functions f (perhaps, bounded and continuous)?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.