Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am interested in computing the margin-of-error for a metric computed on a random sample. The underlying distribution (finite) from which the random sampling is done is not normal (its extremely skewed; something like a zipf/power-law curve).

Planning on estimating the margin-of-error at 95% confidence by Monte Carlo Simulation:

  • Repeat large number of times:
    • Sample $n$ data points (without replacement) out of $N$
    • Compute metric on $n$ points and store it somewhere
  • Use the large number of metrics to estimate margin of error

Question1 :: I am not sure about the sampling without replacement part (should it be with replacement)?

Question2 :: How to compute the margin-of-error? (I can get the histogram, but then what?)


Edit: The metric I am concerned with is "defect rate" in %tage. The reason for sampling is to lower the cost of evaluation of item.

share|improve this question
    
Are you computing a metric on a tuple of $n$ points, or on $n$ individual points and then averaging? In the former case, it's you who decides whether the tuple should allow replacement or not, since you define the metric; in the latter case, your approach of bunching the sample into groups of $n$ is a distraction and you should just take a single very big sample. Also you should say something about whether your samples are correlated (which they usually are in Monte Carlo simulations). –  joriki Jan 17 '13 at 9:41
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.