# Derivation of likelihood equations for a multiple linear regresion, diffrentiating with vectors.

Consider $$Y_i \sim N(\mu_i,\sigma^2)$$ for $$i=1,2,\dots,n$$ all independent where $$\mu_i = \beta_0 +\beta_1 x_{1i} +\dots +\beta_{p-1} x_{p-1i} = \underline{ \beta} \space \underline{x_{i}}$$ where $$\underline{\beta}=(\beta_0 \space \beta_1 \space \dots \space \beta_{p-1})$$ and $$\underline{x_i} = (1 \space x_{1i} \space x_{2i} \space \dots \space x_{p-1i})^T$$.

After some work, we find the log-likelihood of this data to be
$$\ell(\underline{\beta},\sigma^2|\underline{y}) = -\frac{n}{2}\ln(2\pi)-\frac{n}{2}\ln(\sigma^2) - \frac{1}{2 \sigma^2} \sum_{i=1}^n(y_i- \underline{\beta} \space\underline{x_i})$$
Note that we have $$\sum_{i=1}^n(y_i- \underline{\beta} \space\underline{x_i}) = (\underline{y}-X\underline{\beta})^T(\underline{y}-X\underline{\beta})$$ Now I have been told that the likelihood equation for $$\underline{\beta}$$ is $$\frac{\partial \ell}{\partial \underline{\beta}} = \frac{1}{\sigma^2}X^T(\underline{y}-X\underline{\beta})$$ I do not understand how I would take derivatives involving the vector $$\underline{\beta}$$, can anyone explain what happens?

Note that we have $$\sum_{i=1}^n(y_i- \underline{\beta} \space\underline{x_i})^{\color{red}{2}} = (\underline{y}-X\underline{\beta})^T(\underline{y}-X\underline{\beta})$$

The way to take the derivative of $$(y-X\beta)^T(y-X\beta)$$ w.r.t. $$\beta$$ is to expand, ignoring the constant $$y^Ty$$:

$$(y-X\beta)^T(y-X\beta) = y^Ty-2y^TX\beta+\beta^TX^TX\beta$$

Now, take the derivative of $$-2y^TX\beta$$ w.r.t. $$\beta$$. The rule for this is that the thing in front becomes transposed: $$-2X^Ty$$

And the derivative of $$\beta^TX^TX\beta$$ w.r.t. $$\beta$$. The rule for this is that multiply by $$2$$, take the middle part, and right multiply by $$\beta$$: $$2X^TX\beta$$.

Putting together with the $$-1/(2\sigma^2)$$ in front, we get

$$-\frac 1 {2\sigma^2}\left(-2X^Ty+2X^TX\beta\right)=\frac 1 {\sigma^2}X^T(y+X\beta)$$