# Total Derivative and Multilinear Functions

So I'm starting to work through Spivak's Calculus on Manifolds and I'm having a little trouble verifying some of the claims made in the book problems. To review:

Given a function $\mathbf{f}:\mathbb{R}^{n}\to\mathbb{R}^m$, we say that $\mathbf{f}$ is differentiable at a point $\mathbf{a}=(a^{1},\ldots,a^{n})\in\mathbb{R}^n$ (considered as a $1\times n$ matrix) if there exists a linear transformation, $D\mathbf{f}(\mathbf{a}):\mathbb{R}^n\to\mathbb{R}^m$, (considered as an $m\times n$ matrix) such that $$\lim\limits_{\mathbf{h}\to\mathbf{a}}\dfrac{||\mathbf{f}(\mathbf{a}+\mathbf{h})-\mathbf{f}(\mathbf{a})-D\mathbf{f}(\mathbf{a})(\mathbf{h})||}{||\mathbf{h}||}=0.$$

$D\mathbf{f}(\mathbf{a})$ is called the total derivative or Jacobian matrix of $\mathbf{f}$ at $\mathbf{a}$ and is unique.

We are then asked to prove that if $\mathbf{f}:\mathbb{R}^n\times\mathbb{R}^m\to\mathbb{R}^p$ is bilinear, then $$\lim\limits_{(\mathbf{h},\mathbf{k})\to\mathbf{0}}\dfrac{||\mathbf{f}(\mathbf{h},\mathbf{k})||}{||(\mathbf{h},\mathbf{k})||}=0.$$

The best approach I could think of was that $||\mathbf{f}(\mathbf{h},\mathbf{k})||=||\mathbf{h}||\cdot||\mathbf{k}||\cdot||\mathbf{f}(\hat{\mathbf{h}},\hat{\mathbf{k}})||$, where $\hat{\mathbf{x}}=\frac{\mathbf{x}}{||\mathbf{x}||}$. We then have that $||\mathbf{h}||\cdot||\mathbf{k}||\leq||\mathbf{h}||^{2}+||\mathbf{k}||^{2}=||(\mathbf{h},\mathbf{k})||^{2}$, which gives us $$\dfrac{||\mathbf{f}(\mathbf{h},\mathbf{k})||}{||(\mathbf{h},\mathbf{k})||}\leq||(\mathbf{h},\mathbf{k})||\cdot||\mathbf{f}(\hat{\mathbf{h}},\hat{\mathbf{k}})||,$$ where $||\hat{\mathbf{h}}||=||\hat{\mathbf{k}}||=1$. And since $||\mathbf{f}(\mathbf{x},\mathbf{y})||\leq||\mathbf{f}(\hat{\mathbf{x}},\mathbf{y})||\cdot||\mathbf{x}||$ (similarly for $\mathbf{y}$) then $f$ is a bounded (continuous) linear transformation and $$||\mathbf{f}||_{x}=\sup\{||\mathbf{f}(\mathbf{x},\mathbf{y})||:||\mathbf{x}||=1\}<\infty$$ $$||\mathbf{f}||_{y}=\sup\{||\mathbf{f}(\mathbf{x},\mathbf{y})||:||\mathbf{y}||=1\}<\infty,$$ which I was hoping would imply that $||\mathbf{f}(\hat{\mathbf{h}},\hat{\mathbf{k}})||<\infty$, and so we'd have the result we're looking for. But I am stuck here.

Assuming this result, I was able to complete the problem and prove that $$D\mathbf{f}(\mathbf{a},\mathbf{b})(\mathbf{x},\mathbf{y})=\mathbf{f}(\mathbf{a},\mathbf{y})+\mathbf{f}(\mathbf{x},\mathbf{b}).$$

The author then proceeds to ask us to prove the following:

Given a multilinear function $\mathbf{f}:\mathbb{R}^{n_1}\times\cdots\times\mathbb{R}^{n_k}\to\mathbb{R}^p$, show that for $\mathbf{h}=(\mathbf{h}_1,\dots,\mathbf{h}_k)$, with $\mathbf{h}_i\in\mathbb{R}^{n_i}$, we have $$\lim\limits_{\mathbf{h}\to\mathbf{0}}\dfrac{||\mathbf{f}(\mathbf{a}_1,\ldots,\mathbf{a}_{i-1},\mathbf{h}_i,\mathbf{a}_{i+1}\ldots,\mathbf{a_{j-1}},\mathbf{h}_j,\mathbf{a}_{j+1},\ldots,\mathbf{a}_k)||}{||\mathbf{h}||}=0,$$ for $i\neq j$. Use this to prove that $$D\mathbf{f}(\mathbf{a}_1,\ldots,\mathbf{a}_k)(\mathbf{x}_1,\ldots,\mathbf{x}_k)=\sum_{i=1}^{k}{\mathbf{f}(\mathbf{a}_1,\ldots,\mathbf{a}_{i-1},\mathbf{x}_i,\mathbf{a}_{i+1}\ldots,\mathbf{a}_k)}.$$

Using the hint from the book of considering the bilinear function, $\mathbf{g}(\mathbf{x},\mathbf{y})=\mathbf{f}(\mathbf{a}_1,\ldots,\mathbf{x},\ldots,\mathbf{y},\ldots,\mathbf{a}_k)$, I was able to show the first part of the problem assuming the above result (which I still can't prove). However I'm now also stuck on how to prove the rest of the question.

For example, consider the specific case with three arguments. Using the fact that $\mathbf{f}$ is multilinear, we have $$\mathbf{f}(\mathbf{a}+\mathbf{h_1},\mathbf{b}+\mathbf{h_2},\mathbf{c}+\mathbf{h_3})-\mathbf{f}(\mathbf{a},\mathbf{b},\mathbf{c})=$$ $$\mathbf{f}(\mathbf{a},\mathbf{b},\mathbf{h_3})+\mathbf{f}(\mathbf{a},\mathbf{h_2},\mathbf{c})+\mathbf{f}(\mathbf{h_1},\mathbf{b},\mathbf{c}) +$$ $$+\mathbf{f}(\mathbf{a},\mathbf{h_2},\mathbf{h_3})+\mathbf{f}(\mathbf{h_1},\mathbf{b},\mathbf{h_3})+\mathbf{f}(\mathbf{h_1},\mathbf{h_2},\mathbf{c})+$$ $$+\mathbf{f}(\mathbf{h_1},\mathbf{h_2},\mathbf{h_3}).$$ The first three terms are what should be $D\mathbf{f}(\mathbf{a},\mathbf{b},\mathbf{c})$ while the next three will disappear in the limit given the first part of the proof. The trouble is how do I control the limit $\lim\limits_{\mathbf{h}\to\mathbf{0}}\frac{||\mathbf{f}(\mathbf{h_1},\mathbf{h_2},\mathbf{h_3})||}{||(\mathbf{h_1},\mathbf{h_2},\mathbf{h_3})||}$? This trouble arises in the general case as well, any time the number of $\mathbf{h_i}$ is more than two in any single term. E.g. how would I control the term $\mathbf{f}(\mathbf{a_1},\mathbf{a_2},\mathbf{h_3},\mathbf{h_4})$, which has two "$h$" and "$a$" terms each?

Thanks for any help. My analysis skills are a little rusty as I've been focusing on passing my algebra qual.

-

this is just a comment to help you to see why $k$-linear functions on finite dimensional spaces are bounded.

to see that bilinear forms in finite dimensional spaces are bounded you can argue like that: (using your notation)

Let ${\bf x}=(x_1,\ldots,x_n)$ and ${\bf y}=(y_1,\ldots,y_m)$ be unit vectors in $\mathbb{R}^n$ and $\mathbb{R}^m$, respectively, then $$\left\|{\bf f}({\bf x},{\bf y})\right\|=\left\|\sum_{i=1}^n\sum_{j=1}^m x_iy_j\ {\bf f}(e_i,e_j)\right\| \leq \max_{i,j}\ \|{\bf f}(e_i,e_j)\|\sum_{i=1}^n\sum_{j=1}^m (x^2_i+y^2_j).$$ If we call $M=\max_{i,j}\ \|{\bf f}(e_i,e_j)\|$ we have from the above inequality that $${\bf f}({\bf x},{\bf y})\leq M\|{\bf x}\|^2\cdot \|{\bf y}\|^2$$ Since ${\bf x}$ and ${\bf y}$ are unit vectors follows that ${\bf f}$ is bounded by $M$. I hope you can extend this for any $k$-linear function.

-
This argument makes use of the Euclidean norm on $\mathbb{R}^n$ and $\mathbb{R}^m$. You can make it more general if you assume that any two norms on a finite dimensional space are equivalent. In this case you have to replace $M$ by possible bigger constant. –  Leandro Sep 19 '11 at 5:29
Or, for any (positive definite) norm, the unit sphere is compact, so $\|f\|$ attains a maximum value $M$ on the unit sphere. –  Neal Sep 19 '11 at 8:59
Moral of The Story: The algebraic structure of vector spaces and their k-linear mappings is really critical to understanding calculus once one gets beyond the real line! –  Mathemagician1234 Sep 20 '11 at 5:30

@Leandro Thanks for the help; that got me going in the right direction. I had trouble verifying your exact inequality, but I don't think it really matters. I got that $$\left\| {\bf f}({\bf x},{\bf y}) \right\|=\left\| \sum_{i=1}^{n}\sum_{j=1}^{m}{x^{i}y^{j}{\bf f}({\bf e}_i,{\bf e}_j)} \right\| \leq \max_{i,j}\left\| {\bf f}({\bf e}_i,{\bf e}_j) \right\|\sum_{i=1}^{n}\sum_{j=1}^{m}{|x^{i}y^{j}|}$$

and that $|x^iy^j|\leq|x^i|^2+|y^j|^2$, hence

$$\left\| {\bf f}({\bf x},{\bf y}) \right\| \leq M \sum_{i=1}^{n}\sum_{j=1}^{m}{|x^{i}|^2+|y^{j}|^2}.$$

But $\sum_{i=1}^{n}\sum_{j=1}^{m}{|x^{i}|^2+|y^{j}|^2} = m\left\| {\bf x} \right\|^2 + n\left\| {\bf y} \right\|^2 = (n+m)$ if ${\bf x}$ and ${\bf y}$ are unit vectors. Hence I have that $\left\| {\bf f}({\bf x},{\bf y}) \right\| \leq M(n+m)$ which still is independent of both $\bf x$ and $\bf y$, so that $\bf f$ is bounded. Also, isn't this bound independent of the norm as well, apart from the value of $M$?

The process is more or less identical for higher-linear functions, with only the max $M$ (defined in the same way) and the product and sum of the spaces' dimensions contributing to the final bound on $\left\| {\bf f}({\bf x}_1,\ldots,{\bf x}_k) \right\|$. This gives me the final bit of information I needed to show that $D{\bf f}({\bf a}_1,\ldots,{\bf a}_k)$ is indeed as the book stated. (It was a bit misleading, I think, to have the first part of the proof only be concerned with the case for when there are exactly two $\mathbf{h}_i$ terms.)

-
Cool! Perhaps I should be more clear in that step, It is just a binomial inequality $x_i\cdot y_j\leq \frac{1}{2}(x^2_i+y^2_j)<(x^2_i+y^2_j)$. Concerning your question about the norm, you right it depends on the norm. But if you prove that they all are equivalent you are fine ! –  Leandro Sep 20 '11 at 3:27
I follow the inequality $|x_{i}y_{j}| \leq (x_i^2+y_j^2)$, but I still don't see how that implies the conclusion ${\bf f}({\bf x},{\bf y}) \leq M \left\|{\bf x} \right\|^{2}\left\|{\bf y} \right\|^{2}$. I'm just not sure how the double summation leads to the product of the two norms. –  Patch Sep 20 '11 at 4:09
Hi Patch, sorry to take so much time to reply you. After reading carefully your comment I realized that I miss the factor $(n+m)$ in the last inequality of my answer. Sorry about that. –  Leandro Sep 23 '11 at 5:56