2
$\begingroup$

Let $(X_1, X_2,...,X_n) \in \mathbb R^n$ have density function $p(x)$.

(1) Find the density of $(U_{(1)},U_{(2)},...,U_{(n)})$, the order statistics from a sample of iid $\mathbb U[0,1]$ (uniform distributions) variables.

(2) Let $E_1, E_2, ..., E_n$ be i.i.d. exponential with density $p(x)=e^{-x}$, where $x>0$ and for $k=1,2,3,...,n+1$, set $S_k=\Sigma_{i=1}^{k} E_i$. Show that $\left(\frac{S_1}{S_{n+1}},\frac{S_2}{S_{n+1}},...\frac{S_n}{S_{n+1}}\right)=_d (U_{(1)},U_{(2)},...,U_{(n)})$.

(3) For $m$ an integer such that $\frac{m}{n} \rightarrow \alpha$ as $n \rightarrow \infty$, take $U_{(m)}$ as the estimate of the $\alpha$ quantile $x_{\alpha}$; for $\mathbb U[0,1]$, we have $x_{\alpha}=\alpha$. The error made in estimating $x_{\alpha}$ is $U_{(m)}-x_{\alpha}$. Use part (2) to find the asymptotic distribution of $\sqrt n (U_{(m)}-x_{\alpha})$.

Progress: Part (1) is easy and I just plugged in the formula of density function of order statistics. However, I have no idea about part (2) and part (3).

$\endgroup$
3
$\begingroup$

If we let $$ \begin{align*} Y_k & = \frac {S_k} {S_{n+1}}, k = 1, 2, \ldots, n \\ Y_{n+1} & = S_{n+1} \end{align*}$$ Then it is not hard to find out the inverse transform: $$ \begin{align*} E_1 & = Y_1Y_{n+1} \\ E_k &= (Y_k - Y_{k-1})Y_{n+1}, k = 2, 3, \ldots, n \\ E_{n+1} & = Y_{n+1} - Y_nY_{n+1} \end{align*}$$ And thus we have the Jacobian Matrix: $$ \begin{bmatrix} y_{n+1} & 0 & 0 & \ldots & 0 & y_1 \\ -y_{n+1} & y_{n+1} & 0 & \ldots & 0 & y_2-y_1 \\ 0 & -y_{n+1} & y_{n+1} & \ldots & 0 & y_3-y_2 \\ \vdots & \vdots & \vdots & \ddots & \vdots &\vdots \\ 0 & 0 & 0 & \ldots & y_{n+1} & y_n - y_{n-1} \\ 0 & 0 & 0 & \ldots & -y_{n+1} & 1 - y_n \\ \end{bmatrix}$$ Since the determinant is unchanged under row addition, the determinant of the above matrix is the same as an upper triangular matrix by successive row additions, and thus the determinant is just the product of the diagonal entries of the resulting matrix, which is $$y_{n+1}^n$$ Eventually, since the joint pdf of $(E_1, E_2, \ldots, E_n, E_{n+1})$ is $$ f_{E_1, E_2, \ldots, E_n, E_{n+1}}(v_1, v_2, \ldots, v_n, v_{n+1}) = \exp\left\{-\sum_{i=1}^{n+1} v_i\right\}, v_i > 0, i = 1, 2, 3, \ldots, n $$ The joint pdf of $(Y_1, Y_2, \ldots, Y_n, Y_{n+1})$ is $$ \begin{align*} & f_{Y_1, Y_2, \ldots, Y_n, Y_{n+1}}(y_1, y_2, \ldots, y_n, y_{n+1}) \\ = ~& \exp\left\{-\left(y_1y_{n+1} + \sum_{k=2}^n(y_k - y_{k-1})y_{n+1} + y_{n+1} - y_ny_{n+1} \right)\right\}y_{n+1}^n \\ = ~& \exp\left\{-y_{n+1}\right\}y_{n+1}^n, 0 < y_1 < y_2 < \ldots < y_n < 1, y_{n+1} > 0 \end{align*}$$ By integrating this joint pdf with respect to $y_{n+1}$ (actually this auxiliary random variable is independent of the remaining), we have $$ \begin{align*} f_{Y_1, Y_2, \ldots, Y_n}(y_1, y_2, \ldots, y_n) &= \int_0^{+\infty} \exp\left\{-y_{n+1}\right\}y_{n+1}^n dy_{n+1} \\ & = \Gamma(n+1) \\ & = n!, 0 < y_1 < y_2 < \ldots < y_n < 1 \end{align*} $$ which has the same joint pdf as the ordered uniforms. For the last part, we can set $$ m = \lfloor n\alpha\rfloor$$ (ceiling is also ok) It is well known that $$ 0 \leq n\alpha - \lfloor\alpha n\rfloor < 1$$ Dividing the inequality by $n$, by squeezing principle, we have $$ \lim_{n\to+\infty} \frac {\lfloor\alpha n\rfloor} {n} = \alpha $$ Let $$ W = S_m = \sum_{i=1}^{\lfloor n\alpha\rfloor}E_i, V = \sum_{i=\lfloor n\alpha\rfloor+1}^{n+1}E_i$$ Note that $$ E[W] = \lfloor n\alpha\rfloor, E[V] = n + 1 - \lfloor n\alpha\rfloor$$ and $U, V$ are independent. Since we have shown that $U_{(m)}$ has the same distribution of $Y_m$, we can consider the asymptotic of $Y_m$ instead. Note $$ Y_m = \frac {U} {U + V}$$ (it is well known that this function of gamma random variables has a beta distribution) and note $$ y = \frac {w} {w+v}, \frac {\partial y} {\partial w} = \frac {v} {(w+v)^2}, \frac {\partial y} {\partial v} = -\frac {w} {(w+v)^2} $$ Therefore we can use the old trick: Taylor expansion about the mean. $$ \begin{align*} Y_m = ~& \frac {\lfloor n\alpha\rfloor} {\lfloor n\alpha\rfloor + n+1 - \lfloor n\alpha\rfloor} + (W - \lfloor n\alpha\rfloor) \frac {n + 1 - \lfloor n\alpha\rfloor} {(n+1)^2} - (V - (n+1 - \lfloor n\alpha\rfloor)) \frac {\lfloor n\alpha\rfloor} {(n+1)^2} \\ & + R \end{align*}$$ where $R$ is the remaining term. Rearranging a little bit, we have $$ \begin{align*} & \sqrt{n}(Y_m - \alpha) \\ = ~& \sqrt{n}\left(\frac {\lfloor n\alpha\rfloor} {n+1} - \alpha \right) + \sqrt{\lfloor n\alpha\rfloor} \left(\frac {W} {\lfloor n\alpha\rfloor} - 1\right) \frac {(n+1-\lfloor n\alpha\rfloor)\lfloor n\alpha\rfloor} {(n+1)^2} \sqrt{\frac {n} {\lfloor n\alpha\rfloor}} \\ ~& - \sqrt{n + 1 - \lfloor n\alpha\rfloor} \left(\frac {V} {n + 1 - \lfloor n\alpha\rfloor} - 1\right) \frac {(n+1-\lfloor n\alpha\rfloor)\lfloor n\alpha\rfloor} {(n+1)^2} \sqrt{\frac {n} {n + 1 - \lfloor n\alpha\rfloor}} \\ ~& + \sqrt{n}R \end{align*}\\ $$ Eventually we can bound it term by term. For the first term, again since $$ \frac {\lfloor n\alpha\rfloor} {n+1} - \alpha = \frac {\lfloor n\alpha\rfloor - n\alpha - \alpha} {n+1}$$ and $ -1 < \lfloor n\alpha\rfloor - n\alpha \leq 0$, therefore $$ -\sqrt{n}\frac {1 + \alpha} {n+1} < \sqrt{n}\left(\frac {\lfloor n\alpha\rfloor} {n+1} - \alpha \right) \leq -\sqrt{n}\frac {\alpha} {n+1}$$ And by squeezing principle again, this goes to zero as $n \to +\infty$. For the second and third term, note by Central Limit Theorem (CLT), $$ \sqrt{\lfloor n\alpha\rfloor} \left(\frac {W} {\lfloor n\alpha\rfloor} - 1\right) \text{ and } \sqrt{n + 1 - \lfloor n\alpha\rfloor} \left(\frac {V} {n + 1 - \lfloor n\alpha\rfloor} - 1\right) $$ both converges to the standard normal distribution. From the previous limit result, $$ \lim_{n\to+\infty} \frac {(n+1-\lfloor n\alpha\rfloor)\lfloor n\alpha\rfloor} {(n+1)^2} = (1 - \alpha)\alpha$$ $$\lim_{n\to+\infty} \sqrt{\frac {n} {\lfloor n\alpha\rfloor}} = \sqrt{\frac {1} {\alpha}}, \lim_{n\to+\infty} \sqrt{\frac {n} {n+1 - \lfloor n\alpha\rfloor}} = \sqrt{\frac {1} {1 - \alpha}} $$ For the remainder term R, it contains the higher power term which contain at least two of the standardized $W, V$, so by CLT they are at least $O_p(n)$, $o_p(\sqrt{n})$ or in other words goes to zero in probability as $n \to +\infty$. Finally, since $W, V$ are independent, we conclude that $\sqrt{n}(Y_m - \alpha)$ also has a asymtotic normal distribution, with mean zero, and variance $$ (1 - \alpha)^2 \alpha + (1 - \alpha)\alpha^2 = (1 - \alpha)\alpha $$ which agrees with the well known result of the sample quantile.

$\endgroup$
  • $\begingroup$ where is part (3)? $\endgroup$ – Topoguy Jan 5 '16 at 2:33
  • $\begingroup$ Just completed the part 3 as well $\endgroup$ – BGM Jan 5 '16 at 5:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.