Probability distribution modeling of discrete differential estimates using exponential family?

Question

For one reason or another I started to estimate histograms of gradients of natural images.

Let us assume that I want to try to fit a function of the exponential family to this data.

For example $$f(t) = C \exp\left(-\sum_{\forall i}\left(\frac{{|t|}}{b_i}\right)^{a_i}\right)$$

To data where we have measured pairs $(t_k,\hat f(t_k)), k = 1,2,\cdots,n$ and we want to estimate $C,a_i,b_i$

So my question is, how can I find some method to try and optimize

$$\min_{C,a_i,b_i}\left\{\sum_{\forall k} (f(t_k) - \hat f(t_k))^2\right\}$$

Here is an estimation I made manually with just one term.

Answer 1

Your question is a problem of non-linear least mean square fitting.

The general principle to solve it is explain in this paper : https://mathworld.wolfram.com/NonlinearLeastSquaresFitting.html and in many books of statistics. They are some specialized softwares and commercial packages.

All involve iterative processes starting from "guessed" values of the parameters. That is the difficulty in practice especially if the parameters are numerous. This might be a cause of failure if the "guessed" parameters are too far from the unknown correct values and/or if a "guessed" range for the parameters is too large.

A non-iterative method can be used to compute not too bad initial values. The principle is explained in this paper : https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales , with numerical examples.

In the present case $$f(t) = C \exp\left(-\sum_{\forall i}\left(\frac{{|t|}}{b_i}\right)^{a_i}\right)$$ with change of variable $$y(t)=\ln(f(t))$$ and forgetting the too low values $|f(t||<f_{min}$ in order to have $f(t)>0$ for the kept values : $$y(t)=\ln(C)-\sum_{\forall i}\left(\frac{{|t|}}{b_i}\right)^{a_i}$$ This is a sum of powers. In the above referenced paper this case is treated in full details only in the simplest case $i=1$. The method is generalisable for higher $i$ with successive numerical integrations. But this is not done in the paper and would require more preliminary studies.

That is why I doesn't propose presently this non-conventional method and mention it only for information.