# Conformability and Matrix Derivatives

What is the formula for the derivative of the product of two matrices when they are different sizes? Further, what is the formula for the derivative of a Hadamard product when the derivatives of the inputs aren't the same size as the inputs?

Here's what I'm thinking: assume an $n\times 1$ vector $x$, a $n\times n$ symmetric matrix $S$, $f(x)=Sw$, $g(x)=w'Sw$, $h(x)=\frac{f(x)}{g(x)}$, and $j(x)=\operatorname{had}(x, h(x))$ where $\operatorname{had}(a, b)$ is the Hadamard product between the two vectors.

For $f(x)$ and $g(x)$ the derivatives are $S$ and $2\cdot Sw$, respectively. One would think that the derivative of $h(x)$ would be $\dfrac{g(x)f'(x)-g'(x)f(x)}{g(x)^2}$ from the quotient rule. However, the numerator is $n\times n$ and the second part is multiplying a $n\times 1$ vector by an $n\times 1$ vector, which doesn't work.

Similarly, I found a formula for the derivative of Hadamard product and applying it to this would give $\operatorname{had}(x, h'(x)) + \operatorname{had}(I, h(x)) =\operatorname{had}(x, h'(x)) + h(x)$. Since $h(x)$ is $n\times 1$, $h'(x)$ must be $n\times n$. However, the Hadamard product of an $n\times 1$ and $n\times n$ matrix doesn't work. Similarly, the opposite is the case in the second term. So far as I can tell, the terms that are $n\times 1$ need to be copied over to become $n\times n$. This would indicate that the proper formula is something like $\operatorname{had}(\operatorname{kron}(x,\operatorname{ones}),h'(x))+\operatorname{had}(I,\operatorname{kron}(h(x),\operatorname{ones}))$, where $I$ is an $n\times n$ identity matrix and ones is a $1\times n$ vector of ones.

I think that's on the right track since I found a matrix calculus guide that connects the problem to Kronecker products. However, I still had the problem of conformability when using their product rule formula.

-
I think that you mean Hadamard... Capital H. – Mariano Suárez-Alvarez Feb 17 '11 at 6:46

He noted that $g'(x)$ is $1 \times n$ not $n \times 1$ so there is no conformability problem with $h'(x)$. Second, the derivative of the Hadamard product $f(x) \circ g(x)$ is $\operatorname{diag}(g(x)) f'(x) + \operatorname{diag}(f(x)) g'(x)$.