Differentiate vector function wrt vector I have a function $\frac{df(\mathbf{y})}{d\mathbf{y}}=\mathbf{y}g(\kappa)$ where $\kappa=||\mathbf{y}||_2$ and $g(\cdot)$ is a scalar function.
Thing is when I differentiate this function I get a scalar, whereas I am expecting a matrix:
$$
\frac{d^2f(\mathbf{y})}{d\mathbf{y}d\mathbf{y}^T}=g(\kappa)+\mathbf{y}\frac{dg(\kappa)}{d\mathbf{y}}\\
=g(\kappa)+\mathbf{y}\frac{dg(\kappa)}{d\kappa}\frac{d\kappa}{d\mathbf{y}^T}
$$
I'm pretty sure I'm using vector calculus wrong especially since the first term in the differential is a scalar. Wondering what am I doing wrong.
 A: Let $y_i$ represent the $i$-th coordinate of the $\mathbf{y}$ vector. Then,
$$
\frac{d^2f}{dy_j\,dy_i} = \frac{d}{dy_j}(\frac{df}{dy_i}) =
\frac{d}{dy_j}\bigg(\hat{y}_i\,g(\kappa) + \mathbf{y}\,g'(\kappa)\,\frac{d\kappa}{dy_i}\bigg)
$$
where $\hat{y}_i$ is the unit vector along the $i$-th direction, presumed independent of the coordinates of the vector $\mathbf{y}$. Performing the next derivative gives:
$$
\frac{d^2f}{dy_j\,dy_i} =
\hat{y}_i\,g'(\kappa)\,\frac{d\kappa}{dy_j} + 
\hat{y}_j\,\,g'(\kappa)\,\frac{d\kappa}{dy_i} + 
\mathbf{y}\,g''(\kappa)\,\frac{d\kappa}{dy_j}\,\frac{d\kappa}{dy_i} + 
\mathbf{y}\,g'(\kappa)\,\frac{d^2\kappa}{dy_j\,dy_i}
$$
or, more simply,
$$
\frac{d^2f}{dy_j\,dy_i} =
\bigg(\hat{y}_i\frac{d\kappa}{dy_j} + 
\hat{y}_j\frac{d\kappa}{dy_i}\bigg)\,g'(\kappa) + 
\mathbf{y}\,\frac{d^2\kappa}{dy_j\,dy_i}\,g'(\kappa) + 
\mathbf{y}\,\frac{d\kappa}{dy_j}\,\frac{d\kappa}{dy_i}\,g''(\kappa)
$$
And that's the $(ij)$-element of the matrix you were expecting. Note that the resulting matrix is symmetric.
A: First, find the differential of $\kappa$ 
$$\eqalign{
 \kappa^2 &= y^Ty \cr
 2\,\kappa\,d\kappa &= 2\,y^Tdy \cr
 d\kappa &= \frac{y^Tdy}{\kappa} \cr
}$$
and of $g(\kappa)$
$$\eqalign{
dg &= g'd\kappa \cr
  &= \frac{g'\,y^Tdy}{\kappa} \cr
}$$
Then let $u=gy$ and find its differential
$$\eqalign{
 du &= g\,dy + y\,dg \cr
  &= g\,dy + y\Big(\frac{g'\,y^Tdy}{\kappa}\Big) \cr
  &= \Big(g\,I + \frac{g'\,yy^T}{\kappa}\Big)\,dy \cr
}$$
Since $du=(\frac{\partial u}{\partial y^T})\,dy$, the derivative of $u$ is
$$\eqalign{
\frac{\partial u}{\partial y^T} &= \,g\,I + \frac{g'}{\kappa}yy^T \cr
}$$
which is the derivative you were asking about. 
Note that both terms in the sum are square matrices.
