# Maximum likelihood and Bayesian Theorem

In Bayesian theorem, $$p(y|x) = \frac{p(x|y)p(y)}{p(x)}$$, and $p(x|y)$ is called the likelihood, and I assume it's just the conditional probability of $x$ given $y$, right?

The maximum likelihood estimation tries to maximize $p(x|y)$, right? Or $p(x|y)$ is a function of some parameters $\theta$, that is $p(x|y; \theta)$, and MLE tries to find the $\theta$ which can maximize $p(x|y)$?

As you say, $p(x|y)$ is the conditional probability density function for observing $X=x$ given $Y=y$, when seen as a function of $x$.

Since it is based on a probability, you have $\displaystyle\int_x p(x|y) dx =1$.

But $p(x|y)$ can also be seen as a function of $y$, namely the likelihood function for $y$ having observed $X=x$. There is no reason to expect its integral with respect to $y$ to be $1$ so it is not a probability density function for $Y$.

You could turn it into a posterior probability density function for $Y$ by combining with a prior for $y$ using Bayes' theorem.

Or you could seek to find the value for $y$ which maximises $p(x|y)$, to give the maximum likelihood estimate of $Y$.

• So if $p(x|y)$ is seen as a function of $y$, then $y$ can be seen as the parameter of the model with respect to $x$, MLE of $y$ is quite plausible, since AFAIK, MLE is used to find solutions of parameters not random variables, right? BUT here $y$ is random variable not parameter, right? Commented Oct 22, 2013 at 7:19
• @loganecolss - that is actually a deep philosophical question and part of the divide between Bayesian statisticians and Frequentest statisticians. Frequentists would regard $y$ as a parameter which, even if unknown, was fixed and so not a random variable. Bayesians would regard $Y$ as being uncertain, with a degree of belief in different possible values given the observations of $X=x$, and so subject to probabilistic reasoning as if it were a random variable. Commented Oct 22, 2013 at 7:27
• Then from the perspective of Bayesian, $Y$ is regarded as a random variable, does it mean we should assign a probabilistic distribution for $Y$? Commented Oct 22, 2013 at 7:39
• You have already done so twice in your statement of Bayes' theorem: $p(y)$ is the prior distribution function for $Y$ while $p(y|x)$ is the posterior distribution function for $Y$ having observed $X=x$ Commented Oct 22, 2013 at 7:47