Let $P$ denote any continuous distribution with density $p$ on $[0,1]$ and $Q$ the uniform distribution on $[0,1]$ whose density is $1$.

Let $X_1,\ldots,X_n$ be $n$ i.i.d samples drawn from the distribution $Q$ and $X_{(1)},\ldots,X_{(n)}$ be their order statistics and let $Y$ be a random variable distributed acccording to $P$. Define weights as: $$ w_i=\frac{p(X_{(i)})}{q(X_{(i)})}= p(X_{(i)}), 1 \leq i \leq n $$ and $$ \tilde{w}_i = \mathbb{P}(Y \in [X_{(i)},X_{(i+1)}))= F_{P}(X_{(i+1)})-F_P(X_{(i)}). $$

I am trying to understand if there exists any notion of similarity or convergence between the random variables $w_i$ and $\tilde{w}_i$ as $n \to \infty$.

Any help towards this would be appreciated.

The reason why I encountered this is because of the observation that if I use a Gaussian Kernel Density estimator with weights $w_i$ and $\tilde{w}_i$ respectively, both the weights are able to fit the pdf $p(x)$ perfectly across a range of distributions $P$ and $Q$. I want to know if this is a manifestation of some underlying fact about the above defined weights. I am attaching the plots for completeness:

enter image description here enter image description here

  • $\begingroup$ That $w_i$ "fits" $p$ is not surprising, since the points $X_{(i)}$ are distributed pretty uniformly. On the other hand, $\tilde w_i\approx p(X_{(i)}) (X_{(i+1)}-X_{(i)})\approx w_i/n$. $\endgroup$ – zhoraster Sep 7 '16 at 14:50
  • $\begingroup$ @zhoraster: That's true. But can you prove some stronger statement such as $n \tilde{w}_i$ converging to $w_i$ with high probability (over samples $X_1,\ldots,X_n)$ or something? $\endgroup$ – pikachuchameleon Sep 7 '16 at 19:51
  • $\begingroup$ The vagueness of my comment merely reflects that of your question. To start with, could you write clearer, in what sense "both weights are able to fit the pdf" and what do you mean by "across a range of distributions $P$ and $\boldsymbol{Q}$" (I thought $Q=U[0,1]$ was fixed). $\endgroup$ – zhoraster Sep 8 '16 at 19:19
  • $\begingroup$ @zhoraster: Correct. I am trying to derive any relation between the above weights for the simpler case $U[0,1]$ which is indeed fixed. However, I for simulations I used different choices of distributions for $P$ and $Q$ and using the above set of weights, the kernel density estimator in both the cases look almost identical to $p(x)$ is what I meant by "fit the pdf". $\endgroup$ – pikachuchameleon Sep 8 '16 at 22:39
  • $\begingroup$ Where do they "look almost identical"? You plot them against $i/n$ or something like that? $\endgroup$ – zhoraster Sep 9 '16 at 4:31

In short, the two weights sequences you define are asymptotically equivalent in probability. I put a sketch of proof, see below.

But I'd recommend to have a closer look at Chapter 7 of [David H.A., Nagaraja H.N. Order Statistics (2003)]. I believe the topics discussed there are pretty close to the current one.

Proposition. $\forall\rho>0\;\;\mathbb{P}(|w_i-\tilde{w}_i|>\rho)\to 0, \;n \to \infty$.


a) A basic fact of order statistics is that $X_{(i)}$ converges in probability to $\frac{(i-1)/n\;+\; i/n}{2}$ as $n\to\infty$.

b) If we denote $\delta_{i+1}=X_{(i+1)}-X_{(i)}$, then it's possible to get $\delta_{i+1}=\textit{o}(1/n)$ and

$$ F_{P}(X_{(i+1)}) = F_P(X_{(i)}) + p(X_{(i)})\cdot\delta_{i+1} + \textit{o}(1/n^2) $$ using the Taylor approximation; the notation here is $p(\bullet)\equiv F^{\prime}_P(\bullet)$.

c) Finally,

$\mathbb{P}(|w_i-\tilde{w}_i|>\rho) = \mathbb{P}(|p(X_{(i)}) - \left(F_{P}(X_{(i+1)}) - F_P(X_{(i)})\right) |>\rho) = \mathbb{P}(|p(X_{(i)})\cdot\left(1-\delta_{i+1} \right) + \textit{o}(1/n^2)|>\rho). $

(We just used b) and then a).) As we see, the left-hand side of the inequality is of order $\textit{o}(1/n)$ (in probability, see a)). Thus, we can choose appropriate (big enough) $n^\ast$ given any positive $\rho$, which means that the probability of the event $\left\{\ldots >\rho\right\}$ tends to zero. This concludes the proof.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.