Let $\boldsymbol{\beta} := (\beta_1, \ldots, \beta_p)^T$ and $\boldsymbol{X}$ be a matrix of dimension $n \times p$. I'd like to compute the derivative

$$\nabla_{\beta} \left( X \beta \right)$$

  • $\begingroup$ This is a linear equation, so it's very straightforward $$\eqalign{&y=X\beta,\,\,\,&dy=X\,d\beta,\,\,\,\,&\frac{\partial y}{\partial\beta}=X}$$ $\endgroup$ – greg Jun 6 '18 at 20:43

$X\beta$ is $n$-dimensional vector with components $$ (X\beta)_i = \sum_{j=1}^{p}X_{i,j}\beta_j, \quad i=1,\dots,n. $$ so each of its $n$ components is function of $\beta_1,\dots,\beta_p$. Partial derivative of this vector with respect to component $\beta_k$ is obtained simply by partial differentiation of its components: $$ \left(\frac{\partial X\beta}{\partial\beta_k}\right)_i = \frac{\partial (X\beta)_i}{\partial\beta_k} = \frac{\partial}{\partial\beta_k}\sum_{j=1}^{p}X_{i,j}\beta_j = X_{i,k}, \quad k = 1,\dots,p. $$ So partial derivatives $\frac{\partial X\beta}{\partial\beta_k}$ are $n$-dimensional vectors. These vectors are actually columns of matrix $\frac{\partial X\beta}{\partial\beta}$ you are looking for: $$ \left(\frac{\partial X\beta}{\partial\beta}\right)_{i,k} = \left(\frac{\partial X\beta}{\partial\beta_k}\right)_i = X_{i,k}, $$ or just $\frac{\partial X\beta}{\partial\beta} = X$ which is intuitively clear.

For example, when $n=1$ you get $$ X\beta = \begin{pmatrix}X_1 & \dots & X_p\end{pmatrix} \begin{pmatrix}\beta_1 \\ \vdots \\ \beta_p\end{pmatrix} = X_1\beta_1+\dots+X_p\beta_p; $$ $$ \frac{\partial X\beta}{\partial\beta} = \begin{pmatrix}\frac{\partial X\beta}{\partial\beta_1} & \dots & \frac{\partial X\beta}{\partial\beta_p}\end{pmatrix} = \begin{pmatrix}X_1 & \dots & X_p\end{pmatrix} = X. $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.