Likelihood Function

How do we write the likelihood function of a problem that consists of more than one x variable?

(In my case it is x1, x2 and x3)

-
You didn't validate my answer, so I guess you are not really satisfied by it. Maybe you should be specific as to what you want. Plus, I have a question: what do you mean with these thre variables? There are two ways this can be interpreted: 1/ These three variables are i.i.d. and their distribution is some $f(x;\theta)$. 2/ You mean these three form a random vector with joint distribution $f(x_1,x_2,x_3;\theta)$ and you want to compute the likelihood function for i.i.d. random vectors, which is how I interpreted your question. – Raskolnikov May 29 '11 at 15:51

Let me do a couple of worked out examples. Suppose one has a sample of 3 independent and identically distributed random variables $X_1, X_2$ and $X_3$. Let's say the distribution is exponential for the sake of being concrete:

$$f(x;\lambda)=\lambda e^{-\lambda x} \;\; , \text{for } \lambda >0 \;.$$

The likelihood function is then

$$\begin{eqnarray}L(\lambda;x_1,x_2,x_3) & = & f(x_1;\lambda)\times f(x_2;\lambda)\times f(x_3;\lambda) \\ & = & \lambda e^{-\lambda x_1}\lambda e^{-\lambda x_2}\lambda e^{-\lambda x_3} \\ & = & \lambda^3 e^{-\lambda (x_1+x_2+x_3)} \end{eqnarray}$$

which is the product of three density functions for the values $x_1,x_2,x_3$ your random variables take in your sample. If from this likelihood function you want to build an estimator for the parameter $\lambda$, you seek the maximum of the likelihood function. It is customary however to work with the loglikelihood function since it is easier to handle:

$$\begin{eqnarray}\log L(\lambda;x_1,x_2,x_3) & = & \log f(x_1;\lambda) + \log f(x_2;\lambda) + \log f(x_3;\lambda) \\ & = & \log\left(\lambda^3 e^{-\lambda (x_1+x_2+x_3)}\right) \\ & = & 3\log\lambda -\lambda (x_1+x_2+x_3)\end{eqnarray}$$

Finding the maximum of this function can be done by computing the derivative with respect to $\lambda$ and putting it equal to zero. This will lead to an equation which you can solve easily in this case:

$$\begin{eqnarray}\frac{d\log L(\lambda;x_1,x_2,x_3)}{d\lambda} & = & 0\\ \frac{d}{d\lambda}\left(3\log\lambda -\lambda (x_1+x_2+x_3)\right) & = & 0 \\ \frac{3}{\lambda} -(x_1+x_2+x_3) & = & 0 \\ \frac{3}{\lambda} & = & x_1+x_2+x_3 \\ \frac{1}{\lambda} & = & \frac{x_1+x_2+x_3}{3} \\ \lambda & = & \frac{3}{x_1+x_2+x_3} \end{eqnarray}$$

If you want to check that this critical point does indeed correspond to a maximum, you can check if the second derivative of the loglikelihood function is negative. I leave that as an exercise to you.

If by three variables, you mean three jointly distributed variables of which you sample $n$ triplets, then the following is what you are looking for:

If your density function is of the form $f(X;\theta)$ with $X=(X_1,X_2,X_3)$ and $\theta$ a vector of parameters, and if you have $n$ data points $x_i=(x_{i,1},x_{i,2},x_{i,3})$, then you just write

$$L(\theta;x_1,\ldots,x_n) = \prod_{i=1}^n f(x_i;\theta) = \prod_{i=1}^n f(x_{i,1},x_{i,2},x_{i,3};\theta) \; .$$

-
How do you go about calculating it? As you are meant to do n choose x. I'm confused as I have more than one x.... – Cherizzle May 28 '11 at 8:47
You can't calculate anything more specifically if you don't specify the density. Not sure what you are looking after. It seems to me you have a problem with the very concept of likelihood function in itself, no? – Raskolnikov May 28 '11 at 8:48
I can do a worked out example if you want. – Raskolnikov May 28 '11 at 8:49
That would help lots thanks :) – Cherizzle May 30 '11 at 2:23
I edited with a worked out example. Tell me if this is what you were looking for. – Raskolnikov May 30 '11 at 9:13