Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a model that actually is learned from some observed samples. Then I use the model to generate several artificial data.

My question is: Which test should I use to test if the data is of the same validity as the original observed data? (If possible, my data is based on extreme value distribution so it will be better if a specific test could be used to test. ) Thank you very much!

share|improve this question

1 Answer 1

I'm not sure to which extent your questions makes sense, but I'm pretty sure the answer I'm about to give is not the one you want.

You can use Kolmogorov-Smirnov (or Anderson-Darling I suppose) to check if your data (i.e. the empirical distribution) is significantly different from your modeled distribution. However, my attitude in general is that if the K-S test is the answer, you probably asked the wrong question.

One problem with this approach is that if you have a lot of data, K-S will probably reject the model even if it's a fairly good model; and if you have little data, K-S will probably accept the model unless it is a particularly bad fit.

However, the truly important question to ask, and which this approach does not address, is (usually, and for whatever question you are originally interested in):

Is there an alternative model which fits the data equally well (or roughly so), but which would otherwise lead you to a different conclusion?

Without knowing the details of what the original question is, it's hard to be more concrete.

share|improve this answer
    
Hi, @Einar Rødland My model is like to construct a model with latent and observed variables(basically to model their correlations) and then I use the observed variables to learn all the necessary parameters. –  Junting Zhu Apr 4 at 14:36
    
The statistical structure is a vine copula, so later I use a sampling method for vines to generate sample data. –  Junting Zhu Apr 4 at 14:37
    
@JuntingZhu: What is the core question that you are trying to answer? I.e. why do you generate random data sets? –  Einar Rødland Apr 4 at 23:44
    
Rodland I am generating the artificial data sets simply because in real world the actual observed data sets are rare. –  Junting Zhu Apr 5 at 5:18
    
@JuntingZhu: So what is the purpose of using generated random data? To check how well your analysis method works, or what the estimation errors are? If so, you will only know from the simulations how well the method works for data that actually come from the model you're using. –  Einar Rødland Apr 5 at 8:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.