Minimising a Loss Function requiring Matrix by Matrix Derivative I am trying to minimise the following cost function with respect to $X$:
$\mathbf{C}(X) = ||{M \cdot X \cdot \mathbf{1}_{N \times 1} - T}||_{2}^{2}$
Here, $M$, $X$, $T$ are matrices of dimensions $a \times b$, $b \times n$ and $a \times 1$ respectively. $\mathbf{1}_{N \times 1}$ is a column vector containing all values as $1$. 
I have done the following so far:
$\begin{align*}
\nabla_X \mathbf{C}(X) &= 2 \nabla_X(M \cdot X \cdot \mathbf{1}_{N \times 1} - T) \cdot (M \cdot X \cdot \mathbf{1}_{N \times 1} - T) \\
&= 2 \nabla_X(M \cdot X \cdot \mathbf{1}_{N \times 1}) \cdot (M \cdot X \cdot \mathbf{1}_{N \times 1} - T)
\end{align*}$
The idea is to perform a gradient descent once I get the gradient. However, I am not able to evaluate $\nabla_X(M \cdot X \cdot \mathbf{1}_{N \times 1})$. It will be really great if someone can help me with this problem.
Thanks
 A: For convenience, define a new variable 
$$\eqalign{
  Y &= M\cdot X\cdot 1-T \cr
 dY &= M\cdot dX\cdot 1 \cr
}$$
Use this new variable and the Frobenius (:) Inner Product to write the cost function, differential and gradient as
$$\eqalign{
 C &= Y:Y \cr\cr
dC &= 2Y:dY \cr
   &= 2Y:M\cdot dX\cdot 1 \cr
   &= 2M^T\cdot Y\cdot 1^T:dX \cr\cr
\frac{\partial C}{\partial X} &= 2M^T\cdot Y\cdot 1^T \cr
 &= 2M^T\cdot (M\cdot X\cdot 1-T)\cdot 1^T \cr
}$$
A: Basically, you require derivative of a vector w.r.t. a matrix. As per wiki, there is no agreement on what the result of differentiation will look like source. Still, here is an ans. Derivative of $AXB$ w.r.t. $X$  is $B^T\otimes A$. Now, you replace $B$ by $I$ and you have your result see second last page .
Edit: It seems that the author of the above linked page missed something to check the matrix conformation. 
As mentioned in matrix calculus
    $d(AXB)=AdXB=(B^T \otimes A)dX$, where $X$ is a vectorized column matrix. As a sanity check, replace $ B=1_{N \times 1}$ in the above. Now, $1_{N \times 1}^T \otimes A_{a \times b}$ has dimension $a \times bN$. $dX$ is a column vector of 1's of size $bN \times 1$. Multiply the two to get your result of consistent dimension.
