I'm coming at this from a finance and engineering background. I don't have the depth of statistical knowledge to infer this one completely myself, I apologize for that. I'll probably need some hand holding through this since the 30 websites about likelihood equations still seem rather Greek to me. HA! can't be the first time someone used that pun on this site!

I'm trying to build some forecasting models for 48 months of financial time series data on more than 50 accounts. For each account I would like to use the previous 48 months to develop a model and then forecast for an additional 12 months into the future. I know, all Models are False, Some are useful. I'm hoping to use AIC to help me infer the model that is the most likely to be useful. Now, this gets more exciting and makes AIC more important when I start trying to add volume data and seasonality as well as alternate models such as logistic growth... but I digress. Basics first. Date is X, Value is Y.

For now I just need to get the basic idea behind a Log Likelihood comparison between a 2nd and 3rd order polynomial. Now, i understand the idea of using a Chi-squared test to compare the differences between the models based on the number of parameters in each model (2 and 3 respectively). But i DON'T understand what the Likelihood equation is for a polynomial! How do I determine the Likelihood equation for a polynomial? For some reason the vertical bar included in all of the examples is throwing me through a loop.

On this site, http://www.spiderfinancial.com/support/documentation/numxl/reference-manual/appendix/appendix-b

AIC = 2k-2*ln(L)

where k is the number of parameters

and ln(L) is the likelihood equation

How do I determine the likelihood equation for a second and third order polynomial?


So it looks like you're a bit confused about what a (log-)likelihood function actually is.

Likelihood functions, in terms of the models you're looking at, relate to the probability density function of the "noise", "error-term", or "forecast error" in your model... not the functional form of the model itself.

Say you assume the following model:

$$X_t = A X_{t-1}^2 + e_t, \quad t = 1, ..., T$$

where $X_t$ is a vector of variables you're trying to and $e_t$ is your forecast error.

Say you assume that each $e_t$ is i.i.d and follows some probability density function, $f(\mu)$, where $\mu$ is some parameter of the density function. (If you're not 100% clear what I mean by density function, then you're probably a bit too far behind to get anything else I'm about to say. If you do have a vague idea, but could use an example, then check out the normal density which has two parameters, mean ($\mu$) and variance ($\sigma^2$).)

The log-likelihood function for this model is then:

$$\log(L_1(\mu)) = \sum_{t=1}^{T} \log f(\mu)$$

Where $L_1$ refers to the likelihood function of model 1 above.

Now, log-likelihood inference refers to the procedure which finds the value of $\mu$ that best-fits the data you observe. Denote that maximum as $\mathcal{l}_1$.

If you do the same for a third, or forth (etc) order polynomial, you'll be left with a number of max-log-likelihood values: $\mathcal{l}_1, \mathcal{l}_2, \mathcal{l}_3$

What AIC does, in a really oversimplified way, is to help you choose which model is best based on the values of $\mathcal{l}_1, \mathcal{l}_2, \mathcal{l}_3$

For what you're doing, I'd say you're probably (or there's a high likelihood) that you're barking up the wrong tree with AIC as a model selection method. There are many, many different ways to go about model selection. If you're interested, I'd highly recommend checking out this machine learning textbook


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.