0
$\begingroup$

Occasionally, one can see the following expression called cost function:

$$J(\theta)=\sum_{i=1}^{m}y^{i}\log(h_\theta(x^{i}))+(1-y^{i})\log(1-h_\theta(x^{i}))$$.

It looks to be me like maximum likelihood expression. I think it is incorrect to refer to maximum likelihood express as a cost function. They are different. Am I right?

Update In my view cost function is squared deviation or something similar $$\sum_i (x_i - y_i)^2$$

Here are a couple of links regarding maximum likelihood called cost function: Cost1 Cost2

$\endgroup$
7
  • $\begingroup$ could you add a reference to your cost function? alternatively add an explanation of what it actually consists $\endgroup$ – user190080 Jul 20 '15 at 0:49
  • $\begingroup$ no they are different only if the random variable model used for the MLE is corresponds EXACTLY to the reality of how the datas were generated. In nearly every case, that probabilistic model is only a model to approximate the way the datas were generated, it's exactly equivalent to give an a priori bigger cost to some values, and lesser to others, which is a cost function. $\endgroup$ – reuns Jul 20 '15 at 0:51
  • 1
    $\begingroup$ and there is a theorem : for every cost function, it exists a probabilistic model such that the minimization of the cost function is equivalent to the maximization of the likelihood given that probabilistic model. thus choosing a cost function is 100% equivalent to choosing a probabilistic model (and thus a MLE) $\endgroup$ – reuns Jul 20 '15 at 0:55
  • $\begingroup$ the quadratic cost functions are the logarithm of the likelihood of some gaussian probabilistic model for the datas. (one indepedant multi dimensional and indentically distributed gaussian variable, correlated in a linear way) $\endgroup$ – reuns Jul 20 '15 at 0:57
  • $\begingroup$ @reuns. Thank you very much! Could you give me a reference to theorem establishing connection between maximum likelihood and cost function. I am not grasping how likelihood is related to Gaussian distribution. It looks like the above expression is from binomial distribution. $\endgroup$ – user1700890 Jul 20 '15 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.