estimation of pdf

when estimating a probability distribution function one may, for instance, gather a set of data and use the hist function in Matlab to get an idea of what's the pdf like. My question is this: if you have no a-priori knowledge of what the pdf should be, how do you determine how many samples are needed? Do you compute the pdf in time and stop when there is (almost) no change?

-

Your intuition is rather close, although the algorithms used in practice are of course a little bit more sophisticated.

First, when you don't have a model that can be described by a finite set of parameters, then one talks about ""non-parametric statistics" (see Wikipedia). This is a little bit misleading: The space of all possible PDF is an infinite dimensional function space. In parametric statistics, one selects a subspace that is described by a finite set of parameters. In non-parametric statistics one does not do that, so one is actually thinking about a problem with an "infinite number of parameters".

Anyway, in a first step you'll have to decide how you get a PDF from your data, like drawing a histogram or, slightly more sophisicated, using kernel density estimation. Next, you'll have to choose a "measure of goodness" for your estimated density, which is usually some kind of metric on the function space of PDFs.

When we say that $f$ is the true PDF, and $f_e(n)$ is the estimation based on a sample of size $n$, we get a "measure of goodness" via $$h(n) := d(f, f_e(n))$$ with some metric $d$ on the space of PDF. Of course we cannot compute $h$, because we don't know $f$. All of these choices are more or less arbitrary from a pure mathematical viewpoint.

Now, $h$ itself will depending on the sample size $n$, and in many interesting situations it can be shown that $$h(n) = O (n^{-x)}$$ with some real number $x \gt 0$. So, in the simplest possible setting, you'll monitor $$h(n_1, n_2) = d(f_e(n_1), d_e(n_2))$$ and stop sampling after $h(n_1, n_2)$ drops below some threshold, which is what you do if you "stop when there is (almost) no change"".

For more details, see Kernel density estimation.

-