Understanding theorem $9.21$ from Rudin -- Partial Derivatives. Let's say that $f: E \subset \mathbb{R}^n \to \mathbb{R}^m$, where $E$ is an open set,  is continuously differentiable if $f'$ is a continuous mapping of $E$ into $L(\mathbb{R}^n,\mathbb{R}^m)$, and denote the set of such functions by $\mathcal{C}'(E)$.
Now, consider the following theorem:

Suppose $\mathbf{f}$ maps an open set $E \subset \mathbb{R}^n$ into
  $\mathbb{R}^m$. Then $f \in \mathcal{C}'(E)$ if and only if the
  partial derivatives $(D_jf_i)$ exist and are continuous on $E$ for $1
> \leq i \leq m$ and $1 \leq j \leq n$. $f \in \mathcal{C}'(E)$ means
  class of continuous differentiable functions with domain $E$.

This is theorem $9.21$ of Rudin 3rd edition (page $219$).
I am having trouble understanding the proof. 
For the $\implies $ side of the proof, I do not understand where they use the assumption that $f \in \mathcal{C}'(E)$ as well as how they form the first inequality.
For the $\Leftarrow$ implication, I definitely do not follow the argument given. Can anyone break down what is going?
I would definitely appreciate help in understand how the proof of this theorem works.
 A: For the $\Rightarrow$ direction, that $f$ is continuously differentiable is used to force that $\|f'(x) - f'(y)\| \to 0$ when $y\to x$. The first inequality is Cauchy-Schwarz inequality.
For the reserve direction, it's really just what Frank Science suggested. Take the simple case when there is only two variables, we have (First moving the $y$-direction, then the $x$-direction)
$$\begin{split}
f(\vec x + \vec h) - f(x) &= f(x_1 + h_1, x_2 + h_2) - f(x_1, x_2)\\
&=  f(x_1 + h_1, x_2 + h_2) - f(x_1+ h_1, x_2) + f(x_1+ h_1, x_2)-  f(x_1, x_2)
\end{split}$$
For the first two terms, treating $x_1 + h_1$ as fixed and use mean value theorem for $y\mapsto f(x_1 + h_1, y)$, there is $\theta_2 \in (0,1)$ (so that $x_2 + \theta_2 h_2 \in (x_2, x_2 + h_2)$ so that 
$$\begin{split}f(x_1 + h_1, x_2 + h_2) - f(x_1+ h_1, x_2) &= D_2 f (x_1+ h_1, x_2 + \theta_2 h_2) (x_2 + h_2 - x_2) \\&= h_2 D_2 f (x_1+ h_1, x_2 + \theta_2 h_2).\end{split}$$
Similarly for the next two terms, there is $\theta_1 \in (0,1)$ so that 
$$f(x_1+ h_1, x_2)-  f(x_1, x_2) = h_1 D_1 f(x_1 + \theta_1 h_1 , x_2).$$
(These two equations are at the bottom of p.219). Now we have 
$$f(\vec x + \vec h) - f(x)  = h_1 D_1 f(x_1 + \theta_1 h_1 , x_2) + h_2 D_2 f (x_1+ h_1, x_2 + \theta_2 h_2).\tag{1}$$
Not I want to change everything on the right to $D_i f(\vec x) = D_if(x_1, x_2)$. To do so I need to use the continuity of $D_if$. Note that both 
$$(x_1 + \theta_1 h_1 , x_2), (x_1+ h_1, x_2 + \theta_2 h_2)$$
are close to $(x_1, x_2)$ when $(h_1, h_2)$ are small. By continuity of $D_if$, one can choose $h_1, h_2$ small so that 
$$(x_1 + \theta_1 h_1 , x_2), (x_1+ h_1, x_2 + \theta_2 h_2) \in S$$
and so 
$$|D_2 f (x_1+ h_1, x_2 + \theta_2 h_2)- D_2 f(x_1, x_2)|, |D_1 f(x_1 + \theta_1 h_1 , x_2)- D_1f(x_1, x_2)| <\epsilon/2. \tag{2}$$
(Equation 42 in the book). Hence if we subtract on both sides of $(1)$ the term
$$ \sum_{i=1}^2 h_i D_i f(x_1,x_2) ,$$
take the absolute value, and then use $(2)$ on the right hand side, we will get
$$\left|f(\vec x +\vec h) - f(\vec x) - \sum_{i=1}^2 h_i D_i f(\vec x)\right|\le \frac{\epsilon }{2} (|h_1| + |h_2|)\le \|\vec h\| \epsilon.$$
Dividing $\|\vec h\|$ on the both sides, we have that $f$ is differentiable, and the derivative $f'(x)$ is given by 
$$f'(x) (\vec h) = \sum_{i=1}^2 h_i D_i f(\vec x).$$
A: I will try to fill in the missing parts of Rudin proof to make it easier for you to follow.
$f \in \mathcal{C}^{'}(E) \implies  D_jf_i$ exist and are continuous for $1<i<m , 1<j<n$.
The assumption $f \in \mathcal{C}^{'}(E)$ is used as in definition 9.20 which shows that $\;\lVert f^{'}(x) - f^{'}(y) \rVert < \varepsilon\;$ if $\lvert x-y\rvert <\delta$.
The other direction uses the continuity of $D_jf$ to establish that the gradient is continuous $\nabla f(x) h= \sum_{j=1}^{n}h_jD_jf(x)$. I will follow Rudin's proof and fill in the missing pieces so it is clear.
$D_jf$ is continuous and by definition of continuity. Fix $\epsilon >0\; , x \in E\;$ Since E is open, there exists a neighborhood $N_r(x) \subset E \iff \lvert x-y \rvert < r$ such that  $$\lvert D_jf(y) - D_jf(x)\rvert < \frac{\varepsilon}{n} \tag 1$$ $for\; y\in N_r(x) \; ,\;1\leq j \leq n $. (The choice of $\frac{\varepsilon}{n}$ will make sense later in the proof).
Define the vector $h = \sum_{j=1}^{n}h_je_j\; , \; \lvert h \rvert < r\;, v_0=0,\; v_k=h_1e_1+...+h_ke_k\; where \; 1\leq k \leq n$. Hence, $\lvert v \rvert < r\;$ and $v_j= v_{j-1}+ h_je_j \implies x + v_j= x +  v_{j-1}+ h_je_j $. 
Note that $f^{'}(x)h = \sum_{j=1}^{n}h_jD_jf(x) \iff f^{'}(x)\sum_{j=1}^{n}h_je_j = \sum_{j=1}^{n}h_jD_jf(x)$. It remains to prove that the linear transformation $f^{'}(x)$ exists. It is clear that it is continuous since its components $D_jf(x)$ are continuous by assumption. 
To prove $f{'}(x)$ exists, let 
$$f(x+h) - f(x) = \sum_{j=1}^{n} [f(x+v_j) - f(x+v_{j-1})] \tag 2$$
It is easy to verify this relationship by substituting the definition of $v_k$.
Since $\lvert v_k \rvert <r \;\forall k\;$ and $N_r(x)$ is convex, there exists a $\theta_j \in (0,1)$ such that $x+v_{j-1}+\theta_j h_j e_j \in (x+v_{j-1} , x+v_j)$. 
By mean value theorem,
\begin{split} 
\lvert f(x+v_j) - f(x+v_{j-1})\rvert e_j&=\lvert D_j f(x+v_{j-1}+\theta_j h_j e_j)||v_j-v_{j-1}|\\  
&=\lvert D_j f(x+v_{j-1}+\theta_j h_j e_j)||h_je_j| 
\end{split}
hence, by taking the summation of the above equation and substituting in (2) 
$$f(x+h) - f(x) = \sum_{j=1}^{n} h_j D_j f(x+v_{j-1}+\theta_j h_j e_j) \tag 3$$
Subtracting $\sum_{j=1}^{n}h_jD_jf(x)$ from both sides of (3) and using (1) 
\begin{split}
|f(x+h) - f(x) -\sum_{j=1}^{n}h_jD_jf(x)| &\leq \sum_{j=1}^{n} h_j D_j f(x+v_{j-1}+\theta_j h_j e_j) - \sum_{j=1}^{n}h_jD_jf(x)\\ 
&\leq \sum_{j=1}^{n} h_j [D_j f(x+v_{j-1}+\theta_j h_j e_j)- D_jf(x)]\\ 
&\leq \frac{1}{n}\sum_{j=1}^{n} h_j |\varepsilon| = h|\varepsilon|
\end{split}
