# Difference between optimization and estimation

I fear this question may be stupid, but yesterday I got an email from an engineer with these sentences in it:

Subset simulation is a sampling method for estimating failure probabilities. The task of estimating a failure probability is not an optimization problem; it is an estimation problem.

I as an old idiotic mathematician always thought that every statistical estimation problem (excluding pathological cases) can be formulated as an optimization problem.

Maybe I am wrong?

Can anybody of you enlighten me?

Thanks

• If estimation was done rationally, it would amount to minimising a loss function under uncertainty. But that is not usually how it works in practice – Henry Jan 30 '18 at 13:49

In optimization you need an error function to minimize. Depending on the estimation problem it may not be possible to formulate an error function, due to for example fundamentally incomplete information.

E.g. without any further context, assuming you're estimating some ground truth probability $p$, then your squared error function for a guess $x$ would be $f(x) = (p-x)^2$, but to compute this function you need to know $p$ in the first place.

• If you cant formulate an error function in estimation, how do you find your estimator? – Karl Jan 30 '18 at 14:08
• @Karl You estimate it :) That is the core of estimation: a fundamental lack of information. – orlp Jan 30 '18 at 14:17
• But you need to have some measure of the quality of your estimation, otherwise use always 1 as estimate. – Karl Jan 30 '18 at 14:23

Estimators can be constructed without optimization when you have some framework which lets you analytically derive, rather than search for, your estimator.

Some examples:

Whereas, you'd call it an optimisation problem if the best known approach is to search for a solution, rather than calculate a solution.

Optimization algorithms often think about the “landscape” of possible solutions. Estimation techniques may not.