0
$\begingroup$

Suppose $W_i=\frac{\delta_i}{\pi_i} S^\prime_i(y_i,\vec\theta)$, where $\delta_i$ is iid Bernoulli r.v, $y_i$ is fixed variable and $\vec \theta$ is a vector of parameters. $S^\prime_i(y_i,\vec\theta)=\frac{\partial}{\partial\vec \theta}S_i(y_i,\vec\theta)$. Suppose the vector $$ S_i(y_i,x_i,\vec\theta) = \begin{bmatrix} y_i-\theta_1 &\\(y_i-\theta_1)^2-\theta^2_2 \end{bmatrix}$$ then

$$\frac{\partial}{\partial \vec \theta} S_{i}(y_i,\vec \theta) \text{ will be a matrix of order } 2 \times 2 \text{ of fixed quantities }.$$ Now $W_i$ is independent but non-identical random matrices because $E(W_i) = E(\delta_i/\pi_i) S^\prime_{i} (y_i,\vec \theta)=S^\prime_{i}(y_i,\vec \theta)$ since $E(\frac{\delta_i}{\pi_i}|\pi_i)=1$ and $y's$ are fixed. I need to show that $$ S_N=\frac{1}{N}\sum_{i=1}^{N} W_i \text{ convergence in probability to } E(S_N) \text{uniformly}. $$ Any quick help would be greatly appreciated.

$\endgroup$
  • $\begingroup$ This question cannot be answered because the descriptions of $\theta$ and $S_i(\theta)$ are unclear. [Why is a "matrix of fixed quanties" indexed by both $\theta$ and $i$?] [How can we verify the "Hence $W_i$ is independent but non-identical"?] $\endgroup$ – Michael Dec 5 '16 at 23:58
  • $\begingroup$ Thanks Michael. I added the detail. Please now have a look. $\endgroup$ – ZAH Dec 6 '16 at 15:02
  • $\begingroup$ Some details seem irrelevant (such as the $\theta$ vector). Yet, you do not specify anything about the $y_i$ sequence. There is no reason for this to converge to anything. It looks like you have a deterministic sequence of matrices $\{M_i\}_{i=1}^{\infty}$ and an i.i.d. Bernoulli sequence $\{\delta_i\}_{i=1}^{\infty}$, and you are studying $\frac{1}{n}\sum_{i=1}^n \delta_i M_i$. Consider the special case when $P[\delta_i=1]=1$ for all $i$. Well, why should the time average $\frac{1}{n}\sum_{i=1}^n M_i$ converge? $\endgroup$ – Michael Dec 6 '16 at 17:28
  • $\begingroup$ Also, it does not make sense to say "convergence in probability to $E[S_N]$" since that is parameterized by $N$. We can converge to a constant or to a random variable, not to a sequence. I assume you really want to study $\frac{1}{n}\sum_{i=1}^n [\delta_i M_i - E[\delta_i M_i]]$. $\endgroup$ – Michael Dec 6 '16 at 17:34
  • $\begingroup$ $E[S_N]=\sum_{i=1}^{N}S^\prime_i(y_i,\theta)$ is constant in my case because for finite population of size N, $E[S_N]$ is a constant. $\frac{1}{N}\sum_{i=1}^{N}[\delta_iM_i-E[[\delta_iM_i]]$ converge to zero in probability uniformly can also serve my purpose as you assumed in your above comment. $\endgroup$ – ZAH Dec 6 '16 at 20:44
0
$\begingroup$

If we have a sequence of (finite sized) matrices $(a_{ij}(n))$ such that $\lim_{n\rightarrow\infty} a_{ij}(n) = 0$ for all entries $(i,j)$, then the entries indeed converge uniformly to 0 (uniform over all finite entries). So you can reduce the problem to scalars.

If $\{X_i\}_{i=1}^{\infty}$ is a random process of mutually independent random variables, and if $\{a_i\}_{i=1}^{\infty}$ is a deterministic sequence of real numbers, then we can define $S_n = \frac{1}{n}\sum_{i=1}^n X_i a_i$. Then $S_n - E[S_n]$ will converge to 0 in probability (as $n\rightarrow \infty)$ under some mild assumptions on the random process $\{X_i\}$ and on the sequence $\{a_i\}$. Just compute the variance of $S_n - E[S_n]$ to derive the properties that you need.

In general, the quantity $S_n - E[S_n]$ will not converge to 0 in any sense. For example, if $\{X_i\}$ is i.i.d. Bernoulli with $P[X_i=0]=P[X_i=1]=1/2$, and if $a_i = 2^i$, you can see that $S_n$ will jump whenever $X_n=1$ and will never converge to anything.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.