1
$\begingroup$

Consider $Y_i \sim N(\mu_i,\sigma^2)$ for $i=1,2,\dots,n$ all independent where $$\mu_i = \beta_0 +\beta_1 x_{1i} +\dots +\beta_{p-1} x_{p-1i} = \underline{ \beta} \space \underline{x_{i}}$$ where $\underline{\beta}=(\beta_0 \space \beta_1 \space \dots \space \beta_{p-1})$ and $\underline{x_i} = (1 \space x_{1i} \space x_{2i} \space \dots \space x_{p-1i})^T$.

After some work, we find the log-likelihood of this data to be
$$\ell(\underline{\beta},\sigma^2|\underline{y}) = -\frac{n}{2}\ln(2\pi)-\frac{n}{2}\ln(\sigma^2) - \frac{1}{2 \sigma^2} \sum_{i=1}^n(y_i- \underline{\beta} \space\underline{x_i}) $$
Note that we have $$\sum_{i=1}^n(y_i- \underline{\beta} \space\underline{x_i}) = (\underline{y}-X\underline{\beta})^T(\underline{y}-X\underline{\beta}) $$ Now I have been told that the likelihood equation for $\underline{\beta}$ is $$\frac{\partial \ell}{\partial \underline{\beta}} = \frac{1}{\sigma^2}X^T(\underline{y}-X\underline{\beta}) $$ I do not understand how I would take derivatives involving the vector $\underline{\beta}$, can anyone explain what happens?

$\endgroup$

1 Answer 1

1
$\begingroup$

Note that we have $$\sum_{i=1}^n(y_i- \underline{\beta} \space\underline{x_i})^{\color{red}{2}} = (\underline{y}-X\underline{\beta})^T(\underline{y}-X\underline{\beta})$$

The way to take the derivative of $(y-X\beta)^T(y-X\beta)$ w.r.t. $\beta$ is to expand, ignoring the constant $y^Ty$:

$$(y-X\beta)^T(y-X\beta) = y^Ty-2y^TX\beta+\beta^TX^TX\beta$$

Now, take the derivative of $-2y^TX\beta$ w.r.t. $\beta$. The rule for this is that the thing in front becomes transposed: $-2X^Ty$

And the derivative of $\beta^TX^TX\beta$ w.r.t. $\beta$. The rule for this is that multiply by $2$, take the middle part, and right multiply by $\beta$: $2X^TX\beta$.

Putting together with the $-1/(2\sigma^2)$ in front, we get

$$-\frac 1 {2\sigma^2}\left(-2X^Ty+2X^TX\beta\right)=\frac 1 {\sigma^2}X^T(y+X\beta)$$

$\endgroup$

You must log in to answer this question.