How to compute the derivative of $\frac{\partial\mathbf{a}_k }{\partial{X}_{ij}}$ when $X = A^TB$ Assume I have 
$$ X = A^TB $$
where $X \in R^{m \times n}, A \in R^{r \times m}, A \in R^{r \times n}$.
Let's define $A = [\mathbf{a}_1, ..., \mathbf{a}_m]$ where $\mathbf{a}_k $ is the $k$-th column of matrix $A$. How to compute the following derivative?
$$\frac{\partial\mathbf{a}_k }{\partial{X}_{ij}}$$
 A: Let
$$\eqalign{
C^T &= B^+ \quad &({\rm pseudoinverse\,of\,} B) \cr
{\mathcal H}_{ijkl} &= \delta_{il}\delta_{jk} \quad &({\rm isotropic\,tensor}) \cr
X_{ji} &= {\mathcal H}_{ijkl}X_{kl} \quad &({\rm index\,notation}) \cr
X^T &= {\mathcal H}:X \quad &({\rm matrix\,notation}) \cr
}$$
Your question assumes that $A$ can be expressed as a function of $X$, so let's do that.
Then we'll calculate the differential and the gradient.
$$\eqalign{
A &= CX^T = C{\mathcal H}:X \cr
dA &= C{\mathcal H}:dX \cr
\frac{\partial A}{\partial X} &= C{\mathcal H} \cr
\frac{\partial A_{ij}}{\partial X_{pq}}
 &= C_{ik}{\mathcal H}_{kjpq} = B^+_{ki}\delta_{kq}\delta_{jp} = B^+_{qi}\delta_{jp}   \cr
}$$
The result is a fourth-order tensor, and these are its component. To express it in terms of the columns of $A$ and $C$, sum over the $i$-index with vectors from the standard basis.
$$\eqalign{
{\mathbf a}_j &= A_{ij}\,{\mathbf e}_i \cr
{\mathbf c}_q &= C_{iq}\,{\mathbf e}_i = B^+_{qi}\,{\mathbf e}_i \cr
\frac{\partial {\mathbf a}_{j}}{\partial X_{pq}}
 &= B^+_{qi}\delta_{jp} {\mathbf e}_i = \delta_{jp}\,{\mathbf c}_q \cr
}$$
