0
$\begingroup$

I'm having trouble understanding what is $\theta$ in Maximum likelihood method. I know $ {x_1, x_2,...,x_n}$ is sample of some population. I watched many videos and read definitions, but still can't understand what $\theta$ in function $$f=(x_i, \theta)$$ represents. Is it a parameter of distribution function of random variables $ {x_1, x_2,...,x_n}$? How do we choose function $f$? Does ML method estimate distribution function based on observations of some population?

I'm sorry if this doesn't make any sense, but I'm having trouble understanding whole concept of this method. Can you please help me? Thank you!

$\endgroup$
1
$\begingroup$
  1. Is it a parameter of distribution function of random variables $X_1,...,X_n$? - Yes.

  2. How do we choose function $f$? - Mostly it is a matter of assumption rather than choice.

  3. Does ML method estimate distribution function based on observations of some population? - To be precise, ML is method for estimation on unknown parameters. In many cases, these parameters determines completely the exact form of your function. Hence, indirectly you can say that you use ML estimators in order to fined the exact form of the distribution of $X$.

In other words, you are assuming that the sample $x_1,...,x_n$ came from some parametric distribution $f(x;\theta)$, while $\theta$ may be one or more unknown parameters that characterize the distribution of $X$. For instance, let $X \sim \mathcal{E}xp(\theta)$ where $\theta$ is unknown constant. If you know $\theta$ you know everything about the distribution of $X$. Hence, in order to estimate $\theta$ you take a sample of $n$ realizations of such distribution and compute $$ \hat{\theta}_{MLE} = 1/\bar{x}_n. $$
Namely, given some particular realization of $X_1,..., X_n$ you'll have a number that will be a point estimator of $\theta$, and thus you'll be able to conduct any computations that you want (expectation, variance, probability of some event, etc.).

$\endgroup$
  • 1
    $\begingroup$ Just to add to the answer above. $\theta$ may represent several parameters. For instance, if $f$ is the normal distribution then this is parameterised by $\mu$ and $\sigma$. As above MLE would provide estimations of both in terms of the $x_i$. $\endgroup$ – user121049 Feb 6 '18 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.