Concrete Example of Maximum Likelihood Estimator

Question

I was reading this article, about how seatgeek creates its algorithm for choosing the optimal seat:

http://chairnerd.seatgeek.com/the-math-behind-ticket-bargains

Most of it is straightforward up until the part where it says "To estimate $\hat{\Theta} $ in the presence of noisy data, we use a method called maximum likelihood estimation." I tried researching more about it, but I still could not find out how this estimator is used to smooth out the data. I have a couple questions in particular.

i) How is the density function ($L(\hat{\Theta} |R)$ ) used? What does $\hat{\theta}_i$, etc. stand for? I'm guessing $\hat{\Theta}$ is the large matrix which is constantly being smoothed out (or is that $R$?) but how does it change?

ii) What does it mean to adjust the parameter?

Maybe if someone could walk through the example they gave ($\Theta =\left( \begin{array}{c} 1\\ \frac 12\\ \frac 14\\ \end{array} \right)$) and show how to use this maximum likelihood process to smooth the data, I would understand it better.

Thank you!

Answer 1

To answer all your questions in detail would take a lot of explanation, and effectively would involve writing a mini-chapter on MLE. So I will address your questions in as narrow a scope as possible, and refer you to any number of texts in statistical inference.

1. The function $L(\Theta \mid R)$ is not a density function; it is a likelihood function that, for a given sample $R$, characterizes the relative likelihood that a particular vector of parameters $\Theta$ could have led to that particular sample being observed. This likelihood is algebraically equal to the joint density $f_R(r \mid \Theta)$ over the parameter space, but they mean different things: the former considers the sample $R$ to be fixed, and expresses the likelihood of $\Theta$ for a fixed $R$; whereas the latter regards the vector of parameters $\Theta$ to be fixed, and gives a density over all possible outcomes of the sample (and this will integrate to $1$ with respect to $R$, but $L$ need not integrate to $1$ with respect to $\Theta$).

2. The $\hat\theta_i$ are individual parameters contained in the vector of parameters $\hat\Theta = (\hat\theta_1, \hat\theta_2, \ldots, \hat\theta_p)$ where $p$ is the number of parameters (in the example, $p = 3$).

3. The "hat" ($\hat{\phantom.}$) symbol above the $\theta$ or $\Theta$ represents an estimate of the true value of the parameter, which is simply $\theta$ or $\Theta$ without the hat. Thus, $\hat\Theta$ is an estimate calculated in some fashion from the sample $R$ that is intended to represent our best guess as to the true value of the parameter $\Theta$, which remains unknown to us.

4. The "adjustment" process is simply the maximization of the likelihood function $L(\Theta \mid R)$ for a fixed $R$. The $\Theta$ that maximizes $L$ is called the MLE of $\Theta$, and is denoted $\hat \Theta$. This estimator is a statistic (as described in item 3 above), being a function of the sample $R$. For simple cases involving parametric distributions and samples that are IID, the MLE is usually obtainable via analytical methods (e.g., differential calculus) and has a nice closed form. In more complicated situations, numerical methods are usually employed to find a maximum of the likelihood function.

The basic, basic idea behind maximum likelihood is that in some sense, our "best" estimate of what the true value of the parameter(s) for our distribution will be the one that yields the greatest likelihood of having observed the data we obtained. While such a maximum is not guaranteed to be unique, or even "best" in other senses, it is frequently a convenient and powerful method for obtaining point estimates for statistical inference.