1
$\begingroup$

I did a linear regression model (OLS) and a spatial autoregressive model (Spatial lag). I read that for comparing these models I need to use the Akaike information criterion (AIC). The formula is the following $$ AIC=-2log L(\hat\theta)+2k $$ where $\theta$ is the vector of model parameters, $L(\hat\theta)$ is the likelihood of the candidate model given the data when evaluated at the maximum likelihood estimate of $\theta$ and $k$ is the number of estimated parameters in the candidate model.

But I don't understand what parameters $\theta$ are necessary to use AIC in a linear regression context. For example, if I have a model like this $Y=\beta_0+\beta_1X_1+\beta_2X_2+...+\beta_nX_n+\epsilon$ where $\epsilon$ is a normal variable with zero mean and variance $\sigma$ How do I calculate its AIC?

$\endgroup$
1
$\begingroup$

The parameters $\theta$ are anything you need to estimate in your model. In the case of OLS, it would be the regression coefficients $\beta_0,\ldots,\beta_n$, so you have $k=n+1$.

The log likelihood for OLS is simply the quadratic function that comes from the Gaussian-nkise assumption: $$\log L(\theta) = -\frac{N}{2}\log \sigma^2-\sum_{k=1}^N -\frac{1}{2\sigma^2}( y_k -\beta_0-\beta_1 x_{k,1} -\cdots -\beta_n x_{k,n})^2 .$$

$\endgroup$
5
  • $\begingroup$ Yea, but $L(\hat \theta)$ is a real number, for example, here math.stackexchange.com/questions/2093369/… he or she took the derivate of $L$ with respect to $\sigma$ (I don't know why ) and then evaluated that value in $L$ to obtain $L(\hat \theta)$ $\endgroup$ – Alexis Galois Jun 1 '19 at 5:30
  • $\begingroup$ That is if $\sigma$ is unknown; then you maximize the likelihood with respect to it to find the maximum likelihood estimate. It's clear that $L(\hat \theta)$ is a real number: adding $k$ to it you get the $AIC$, which is itself a real number. $\endgroup$ – Riccardo Sven Risuleo Jun 1 '19 at 8:21
  • $\begingroup$ So, to apply the AIC to a linear regression I have to assume that $Y$ are iid with normal distribution to get its likelihood function, right? $\endgroup$ – Alexis Galois Jun 1 '19 at 22:30
  • $\begingroup$ If $\epsilon$ is normal and $X$ are given, then $Y$ is necessarily normal :) but yes, the likelihood function is based on some assumption on the distribution of the observations (in this case, normal). $\endgroup$ – Riccardo Sven Risuleo Jun 2 '19 at 5:58
  • $\begingroup$ Great!! I have understood everything thank you very much. $\endgroup$ – Alexis Galois Jun 2 '19 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.