5
$\begingroup$

I read in several articles that a model $\{p_\theta , \theta \in \Theta\}$ is well-specified if there exists a $\theta^* \in \Theta$ such that $p_{\theta^*} = p^*$.

Similarly, a model $\{p_\theta , \theta \in \Theta\}$ is misspecified if there does not exists a $\theta^* \in \Theta$ such that $p_{\theta^*} = p^*$.

Here, the definition of $p^*$ is that it is the true "data-generating distribution".

I can understand the terminology but what is the underlying logic here? I feel that I am missing the deeper story here. What is an example of a misspecified model? thanks!

$\endgroup$
1
  • $\begingroup$ If your model is misspecified, then you'll never learn the true data-generating distribution no matter how many examples you see. A simple example is if your model is that your data is being generated by a linear function with some parameters but in fact it is not linear at all. $\endgroup$ Mar 6, 2017 at 23:31

1 Answer 1

7
$\begingroup$

Well-specified means that the class of distribution $\mathcal{C}$ you are assuming for your modeling actually contains the unknown probability distribution $p$ from where the sample is drawn.

Misspecified means, on the other hand, that $\mathcal{C}$ does not contain $p$. You made a modeling assumption, and it is not perfect: for instance, you assume your sample is Gaussian, but (maybe due to noise, or just inherently) it is not actually originating from any Gaussian distribution. It may be close to some Gaussian, if you're lucky, but not exactly.

For instance, many papers dealing with learning the structure of data (think of social sciences, biology, physics) may assume some underlying structure: "the data is coming from some multidimensional Gaussian," or "the different coordinates are independent (product distribution)." This is a modelling assumption which simplifies a lot and typically avoids intractability or impossibility to reasonably analyze the data; but it's most likely only an approximation of the real, actual data. This is what a misspecified model is: a convenient (or even necessary) assumption on the distribution underlying the data, which may only be a reasonable approximation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.