I am stuck with seemingly simple chain rule application. Consider this vector valued function:

$f(\alpha,\beta)=\begin{pmatrix}\alpha^{2} \\ -\beta\end{pmatrix}$

Now I need to compute following derivative:

$$\frac{\partial}{\partial \alpha}(f\circ f)'$$

According to chain rule, that should be something like:

$f'(f))\cdot f'=\begin{pmatrix}2f \\ -1\end{pmatrix}\cdot\begin{pmatrix}2 \\ -1\end{pmatrix}$

However, this results in multiplying two column vectors together, which is not possible.

Where am I doing the mistake? Any help would be appreciated.

Update (more complicated case):

I have functions that map 2D points to another 2D points, the point is defined as:

$\mathbb{x}=\begin{pmatrix}x \\ y\end{pmatrix}$

Now there is one linear mapping:

$P(\mathbb{x}|a,b,c,d)=\begin{pmatrix}a & b \\ c & d\end{pmatrix}\mathbb{x}$

And one non-linear:

$L(\mathbb{x}|\kappa)=\begin{pmatrix}x^{2}\kappa \\ xy\kappa\end{pmatrix}$

Now how to compute gradient of the function:

$p=L(P\mathbb{x})$ ?

the gradient should be:

$\nabla p=\begin{pmatrix}\frac{\partial}{\partial a} \\ \frac{\partial}{\partial b} \\ \vdots \\ \frac{\partial}{\partial \kappa} \end{pmatrix}$

now the single element of the gradient should be computable using chain rule, e.g.:

$\frac{\partial}{\partial a}p=\frac{\partial}{\partial a}L(P\mathbb{x})\cdot \frac{\partial}{\partial a}P\mathbb{x}$

The first element of the product will become 2-vector and the second also...

Furthermore, if I go for total derivatives, since there are only 2 coordinates (x,y) and 5 variables, the Jacobians would be $2\times 5$ and therefore cannot be multiplied together.

  • $\begingroup$ Your map is $f:\mathbb{R} \to \mathbb{R}^2$, so how can you compose $f$ with itself? $\endgroup$ – Mnifldz Feb 2 '15 at 7:13
  • $\begingroup$ @Mnifldz I have adjusted the question. I am really stuck with this entire day - all the tutorials show just simple examples and not when we are composing two or more functions that map vectors to vectors. $\endgroup$ – Libor Feb 2 '15 at 7:20
  • 2
    $\begingroup$ To compose two functions, the output of one must be 'compatible' with the input of the other. $\endgroup$ – copper.hat Feb 2 '15 at 7:21
  • 1
    $\begingroup$ $f'(f)$ should be a matrix as well. Whenever you have a function $\mathbb R^2\to\mathbb R^2$, it's derivative will be a 2x2 matrix. $\endgroup$ – Dom Feb 3 '15 at 0:29
  • 1
    $\begingroup$ I haven't used the chain rule above, but to do that you would take $f'(f(\alpha,\beta))=\left(\begin{matrix}2\alpha^2&0\\0&-1\end{matrix}\right)$. $\endgroup$ – Dom Feb 3 '15 at 0:44

I will do out the example you mention in this third version of your question.

$P$ is a function $P:\mathbb{R}^6\to\mathbb{R}^2$ defined by $$P\begin{pmatrix} x\\y\\a\\b\\c\\d \end{pmatrix} =\begin{pmatrix}a&b\\c&d\end{pmatrix}\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix} ax+by\\cx+dy \end{pmatrix}$$ $L$ is a function $:\mathbb{R}^3\to\mathbb{R}^2$ defined by $$L\begin{pmatrix} \kappa\\x\\y \end{pmatrix}=\begin{pmatrix} x^2\kappa\\xy\kappa \end{pmatrix}$$ Therefore $L$ and $P$ cannot be composed. However, for any given value of $\kappa$ you can form the function $L_\kappa:\mathbb{R}^2\to\mathbb{R}^2$; for example if $\kappa=3$, then the function $L_3$ is defined by $$L_3\begin{pmatrix} x\\y\end{pmatrix}= \begin{pmatrix} 3x^2\\ 3xy \end{pmatrix}$$ Then $P$ can be composed with any choice of these functions to form $(L_\kappa\circ P):\mathbb{R}^6\to\mathbb{R}^2$.

Here are the derivatives: (I am using the $0$ subscripts to encourage you to think of them all as fixed numbers.)

$$P'\begin{pmatrix} x_0\\y_0\\a_0\\b_0\\c_0\\d_0 \end{pmatrix} =\begin{pmatrix} a_0 & b_0 & x_0 & y_0 & 0 & 0\\ c_0 & d_0 & 0 & 0 & x_0 & y_0 \end{pmatrix}$$ For any given $\kappa_0$, $$L_{\kappa_0}'\begin{pmatrix} x_0\\ y_0 \end{pmatrix}=\begin{pmatrix} 2\kappa_0x_0 & 0\\ \kappa_0y_0 & \kappa_0x_0 \end{pmatrix}$$ Therefore $$\begin{align*} (L_{\kappa_0}\circ P)'\begin{pmatrix} x_0\\y_0\\a_0\\b_0\\c_0\\d_0 \end{pmatrix}&=L_{\kappa_0}'\left(P\begin{pmatrix} x_0\\y_0\\a_0\\b_0\\c_0\\d_0 \end{pmatrix}\right)\cdot P'\begin{pmatrix} x_0\\y_0\\a_0\\b_0\\c_0\\d_0 \end{pmatrix}\\\\\\ &=L_{\kappa_0}'\begin{pmatrix} a_0x_0+b_0y_0\\c_0x_0+d_0y_0 \end{pmatrix}\cdot P'\begin{pmatrix} x_0\\y_0\\a_0\\b_0\\c_0\\d_0 \end{pmatrix}\\\\\\ &=\underbrace{\begin{pmatrix} 2\kappa_0(a_0x_0+b_0y_0) & 0\\ \kappa_0(c_0x_0+d_0y_0) & \kappa_0(a_0x_0+b_0y_0) \end{pmatrix}}_{\text{a }2\times 2\text{ matrix}}\underbrace{\begin{pmatrix} a_0 & b_0 & x_0 & y_0 & 0 & 0\\ c_0 & d_0 & 0 & 0 & x_0 & y_0 \end{pmatrix}}_{\text{a }2\times 6\text{ matrix}} \end{align*}$$

  • $\begingroup$ @Sal Thanks for your tremendous help. I think I will work this out from now on. $\endgroup$ – Libor Feb 2 '15 at 9:17
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – Daniel Fischer Feb 2 '15 at 13:17
  • $\begingroup$ @Sal I have added a new question with the exact sample and my computations - although it seems all formulas are used correctly, the results are different: math.stackexchange.com/questions/1130699/… $\endgroup$ – Libor Feb 2 '15 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.