# Generalizing the product rule to dot products

I will be using the notation, assumptions, and definitions mentioned in the statement of this question: Proving that a vector-valued function is differentiable

Define $\varphi \bullet \psi:\Omega\rightarrow\mathbb{R}$ by $(\varphi \bullet \psi)(\underline{x})=\varphi(\underline{x})\bullet\psi(\underline{x})$, where "$\bullet$" denotes a dot product. We also define $\mu:\mathbb{R}^{2p}\rightarrow\mathbb{R}$ by writing elements of $\mathbb{R}^{2p}$ as $\begin{bmatrix}\underline{x} \\ \underline{y} \\ \end{bmatrix}$ for $\underline{x},\underline{y}\in \mathbb{R}^p$, and letting $\mu(\begin{bmatrix}\underline{x} \\ \underline{y} \\ \end{bmatrix})= \underline{x} \bullet \underline{y}$.

Prove that $\varphi \bullet \psi$ is differentiable at $\underline{a}$, and find an expression for its differential.

Like the title says, I'm pretty sure this is leading to the generalization of the product rule, and so I expect the answer to be $d(\varphi \bullet \psi)(\underline{a})=d\varphi(\underline{a}) \bullet \psi(\underline{a})+\varphi(\underline{a}) \bullet d\psi(\underline{a})$.

We have already shown that $\varphi \bullet \psi=\mu \circ \begin{bmatrix}\varphi \\ \psi \\ \end{bmatrix}$, and we've proven the chain rule as well:

$d(\varphi \circ \psi)(\underline{a})=d\varphi(\psi(\underline{a}))\circ d\psi(\underline{a})$

I'm just not sure where to begin (or how to finish). I think I should try to show that $\mu$ is differentiable, and then by the chain rule I at least know the dot product function is differentiable. How I can arrive at the desired product rule is a bit of a mystery to me, though.

## 1 Answer

We don't need the $\mu$. You are given a scalar-valued function $$f(x):=\phi(x)\cdot\psi(x)$$ and are asking for its derivative. The derivative of $f$ at a point $a$ either appears as a linear functional $df: \>{\Bbb R}^n\to{\mathbb R}$ or as gradient vector $\nabla f(a)$. You are right in conjecturing that somehow the product rule applies, but writing $df$ in the form you tried makes no sense: You can not take the scalar product of a linear map $d\phi(a): \>{\Bbb R}^n\to{\Bbb R}^n$ with a vector.

In order to see what's going on here we have to go back to the definitions. We have \eqalign{f(a+h)-f(a) &= \bigl(\phi(a+h)-\phi(a)\bigr)\cdot\psi(a)+\phi(a)\cdot\bigl(\psi(a+h)-\psi(a)\bigr)\cr &\qquad+ \bigl(\phi(a+h)-\phi(a)\bigr)(\psi(a+h)-\psi(a)\bigr) \ . \cr} Since $$\phi(a+h)-\phi(a)=d\phi(a).h+R(h)$$ with $R(h)/|h|\to0$, and of course $|\phi(a+h)-\phi(a)|\leq M|h|$ for suitable $M$ we get \eqalign{&{\bigl|f(a+h)-f(a)-d\phi(a).h \cdot\psi(a)-\phi(a)\cdot d\psi(a).h\bigr| \over |h|}\cr &\qquad\qquad\leq{|R_1(h)|\over|h|}|\psi(a)|+{|R_2(h)|\over|h|}|\phi(a)|+M_1M_2|h|\ .\cr} Here the right hand side converges to $0$ when $h\to0$. This proves that $$df(a).h=d\phi(a).h \cdot\psi(a)+\phi(a)\cdot d\psi(a).h \ .\tag{1}$$ Now in an inner product space we have the rule $$y\cdot Ah= A^\top y\>\cdot h\ .$$ Therefore we can replace $(1)$ by $$df(a).h=d\phi(a)^\top\!. \psi(a)\cdot h +d\psi(a)^\top\!.\phi(a)\cdot h \ ,$$ and this shows that $$\nabla f(a)=d\phi(a)^\top\!. \psi(a) +d\psi(a)^\top\!.\phi(a)\ .$$

• Thank you for your response. However, I have not seen the $o(|h|)$ notation before. I can see why your argument will work, but is there a way to frame it in terms of limits? – Ducky Feb 4 '15 at 14:13