Explanation of maximum likelihood While revisiting maximum likelihood notes i got confused about maximum likelihood and probability density function.
It is saying that 
We assume that the examples are independent, so the probability of the set is
the product of the probabilities of the individual examples:
$$
f(x_1,...x_n;\theta)=\prod\limits_{j} f_\theta(x_j;\theta)\\
$$
The notation above makes us think of the distribution $\theta$ as fixed and the examples
$ x_j $ as unknown, or varying. However, we can think of the training data as fixed
and consider alternative parameter values. This is the point of view behind the
definition of the likelihood function:
$$L(\theta;x_1,...x_n) = f(x_1,...x_n;\theta)$$
Note that 


*

*if $ f(x; \theta) $ is a probability mass function, then the likelihood is always
less than one. 

*if $ f(x; \theta) $ is a probability density function, then the likelihood
can be greater than one, since densities can be greater than one.
Can someone explain 
if $ f(x; \theta)$  is a probability density function then how likelihood will be greater than one ? 
 A: Note that $f(x;\theta)$ can be larger than one, but does not have to be larger than one. 
The explanation is simple: densities are basically real functions which integrate to one. But it is not necessary that a density function $f$ satisfies $f(x) \leq 1$ for all $x\in \Bbb R$. For example the density of a continuous uniformly distributed random variable on $[0, \frac 12]$ is given by
$$
f(x) = \begin{cases}2, & x\in [0,\frac 12], \\
0, &x\notin [0,\frac 12]. \end{cases}
$$ 
A: This example illustrates the role of a discrete PDF in finding probabilties,
and (considered as a likelihood function) in estimating a parameter. Specific comments
about the 'heights' of PDFs and likelihood functions are shown
in italics.
Suppose that you observe $X \sim Binom(n, p).$ Then the PDF consists
of individual probabilities $f(x;p) = P(X = x) = {n \choose x}p^x (1-p)^{n-x},$
for $x = 0, 1, \dots, n.$ In this case the probabilities all add to $1,$ so none of them can exceed 1. 
Finding probabilities. If $p$ is known, the $f(x;p)$ gives a way to evaluate
$P(X = k).$ For example, if $n = 10$ and $p = .4,$ we can use this formula
to find $P(X = 3).$ In R statistical software, this probability is found
to be 0.3823 (height of the red line in the figure below), but a hand calculation would not be difficult.
 pbinom(3, 10, .4)
 ## 0.3822806


Estimating $p$. However, if $X = x$ is observed and we want to estimate $p,$ then we can view
$f(x;p)$ as a function of $p$, calling it a 'likelihood function'. One way
to estimate $p$ is to find the value $\hat p$ at which $f(x;p)$ is a maximum.
Suppose $n  = 10$ and we observe $X = 6.$ We can use R to sketch a graph of the likelihood function as follows:
 p = seq(0,1, by=.001)
 like = dbinom(6, 10, p)
 p.hat = p[like=max(like)];  p.hat
 ## 0.6
 plot(p, like, type="l", lwd=2, col="blue")
 abline(v=p.hat, col="red");  abline(h=0, col="green2")


We say that $\hat p = 0.6$ is the maximum likelihood estimate (MLE) of $p.$
The code above searches for the maximizing value of $p,$ but in this case it is easy to
find the maximizing value using calculus. 
In this process, we are using the likelihood curve only to find its maximum.
For that, it is not necessary to include the constant factor ${10 \choose 6}$
as part of the likelihood function, and so the true height of the likelihood
curve is not an issue. 
Many authors stipulate that a likelihood function
is defined only up to a constant multiple, and use the proportionality
symbol $\propto$ accordingly: $f(x;p) \propto p^x(1-p)^{n-x}.$ This is
especially common in Bayesian applications.
If $X$ has a continuous distribution, then the density function can exceed $1,$ as shown in the Answer of @Cettt (+1).
Note: The estimate $\hat p = 0.6$ above is hardly a surprise. The method
of moments gives the same value: $E(X) = np$ so the MME is $X/n = 6/10.$ 
