0
$\begingroup$

I have a function $f: R^{n \times d} \mapsto R$. It's matrix input (say, $G \in R^{n \times d}$) in turn is a linear function of a $d \times 1$ vector $v$. $G = HK$, where $H \in R^{n \times n}$ is a matrix of constants and the first $(n-1)$ rows of $K \in R^{n \times d}$ are constants and the last row is $v^T \in R^{1 \times d}$. I need to find a vector $d \times 1$, which is the derivative of $f$ w.r.t. $v$, i.e., $\frac{\partial f}{\partial v}$. I tried writing it as: $\frac{\partial f}{\partial v} = \frac{\partial f}{\partial G} \frac{\partial G}{\partial v}$. I can get $\frac{\partial f}{\partial G}$, which is $R^{n \times d}$ but I get stuck computing $\frac{\partial G}{\partial v}$. It turns out to be a $n \times d \times d$ tensor. Each $n \times d$ slice of the tensor has only one non-zero column. I am not sure, how to arrive at the final $d \times 1$ (or $1 \times d$) vector, which is $\frac{\partial f}{\partial v}$?

$\endgroup$
1
$\begingroup$

You haven't provided any information about the function $f$ in your problem statement, but I'll assume you know how to calculate its gradient, i.e. $$A=\frac{\partial f}{\partial G}$$ and don't need any help with that term.

To avoid the need to calculate intermediate quantities like $\frac{\partial G}{\partial v}$ (which is a 3rd order tensor by the way), let's proceed using the differential (which is simply a matrix). $$\eqalign{ df &= A:dG \cr &= A:H\,dK \cr &= H^TA:dK \cr \frac{\partial f}{\partial K} &= H^TA = H^T\frac{\partial f}{\partial G} \cr }$$ We have found the gradient wrt the entire $K$ matrix, but you only wanted to know about the last row. Multiply the result by the cartesian basis vector $e_n$ since $(v^T = e_n^TK)$ yields $$\eqalign{ \frac{\partial f}{\partial v} &= e_n^T\frac{\partial f}{\partial K} = e_n^TH^T\frac{\partial f}{\partial G} \cr }$$ In some of the steps above, I used a colon to denote the trace/Frobenius product $$A:B={\rm tr}(A^TB)$$

$\endgroup$
  • $\begingroup$ Thank you for the answer. $f$ is a function for which the gradient w.r.t. to its matrix argument can be calculated. I already have this result in explicit form. $\endgroup$ – haripkannan Mar 24 '18 at 12:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.