As a word of background, I want to show that certain result is stable when averaging over a large number of simulations, but could be just a lucky draw with a small number of simulations.

I have a computationally expensive, complicated stochastic process and I am bootstrapping the mean result over a number of simulations - in particular I am finding the bootstrap confidence interval for this mean. My bootstrap confidence interval is stable when I run a lot (say, $10^5$) simulations and draw with replacement large samples (again, $10^5$). I would like to find the confidence intervals for the small sample mean (say, $10^3$).

For simplicity of the MWE, assume the generating process is just a random draw from $\mathcal{N}(0,1)$:

fboot <- function(a,i) mean(a[i])
iterations <- 10
sim_results <- apply(matrix(rnorm(iterations^2,0,1),iterations,iterations),2,mean)
b <- boot(sim_results, fboot, R=1000)
boot.ci(b, conf=0.95, type="norm")

How do I find the CI for

iterations <- 10

when every time I run the above I get completely different ones because my simulated sample is so small? I understand I can increase the number of iterations in the simulation, but this will give me the CI for a larger bootstrap draw - I want the bootstrap draw to be size 10.

  • $\begingroup$ I feel I could help you if only I understood what you asked. Are you using only 10 bootstrap samples? If so, why? Also, the code is not clear to me? What language is it? $\endgroup$ – Martijn Pot Apr 10 '15 at 19:33
  • $\begingroup$ The language is R. The number of bootstrap samples is R=1000 - this is not what I am concerned with. I want to find a confidence interval of the sample mean, where my sample is size 10. I am not sure how to make sure I get stable CI - a bootstrap on the original sample size 10 will not describe the distribution very well. $\endgroup$ – adrug Apr 10 '15 at 19:46
  • $\begingroup$ Do you want 1000 simulations as you state in the text, or 10 simulations as you state in your code? $\endgroup$ – Martijn Pot Apr 10 '15 at 19:53
  • $\begingroup$ I want to show the difference in the size of the confidence interval - I want to show that it's wide when I average the results of 10 simulations, but narrow when I average the results of 1000 simulations. The issue is the size-10 confidence interval gets moved around a lot depending on the original draw. $\endgroup$ – adrug Apr 10 '15 at 19:55
  • $\begingroup$ The simulations result should be a good enough representation of the distribution you are sampling with your simulations. Maybe you could do a sort of two stage bootstrap. First calculate e.g. 2 CI's using 2 halves of the simulations result. If it is similar, than you can trust the result of a bootstrap of all simulations result. $\endgroup$ – Martijn Pot Apr 10 '15 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.