Total derivative for a polynomial I refer to Rudin's (Principles of Mathematical analysis, 3rd ed.) definition of differentiability: 
Suppose E is an open set in $R^n$ and f maps E into $R^m$ and $x \in E$.
If there exists a linear transformation A of $R^n$ into $R^m$ such that 
$$\lim_{|h| \rightarrow 0} \frac{ |f(x+h) - f(x) -Ah|  } {|h|} = 0,$$
then we say that f is differentiable at x and we write 
$f'(x) = A$.
With this definition and assuming the $l_2$ norm, $f'(x)$ for $(x_1^2 + x_2^2)$ is given by $(2x_1, 2x_2)$. 
since 
$$\lim_{|h| \rightarrow 0} \frac{ |f(x+h) - f(x)|  } {|h|} =  \lim_{|h| \rightarrow 0} \frac{2(x_1h_1 + x_2 h_2)}{|h|} = \lim_{|h| \rightarrow 0} \frac{(2x_1, 2x_2)'(h_1, h_2)}{|h|}.$$
However I could not find $f'(x)$ for $(x_1 + x_2)^2$ with this definition. 
I get 
$$\lim_{|h| \rightarrow 0} \frac{ |f(x+h) - f(x)|  } {|h|} =  \lim_{|h| \rightarrow 0} \frac{2(x_1 + x_2)(h_1 + h_2) + h_1h_2}{|h|}$$
The cross term $h_1 h_2$ becomes a problem, if it were not present, I would write $f'(x) = (2(x_1+x_2), 2(x_1+x_2))$.
I have read elsewhere on this forum that polynomials are differentiable. I would be very thankful I someone were to tell me how do I proceed to find the total derivative, $f'(x)$ for a given polynomial function.
 A: Caution: It isn't correct to write
$$
f'(x) = \lim_{h \to 0} \frac{\left|f(x + h) - f(x)\right|}{\left|h\right|}
$$
even for functions of one variable, and it isn't sensible to write
$$
f'(x) = \lim_{h \to 0} \frac{f(x + h) - f(x)}{h}
$$
for functions of more than one variable, since division by a vector is generally meaningless.
Instead, you must write the condition for multivariable differentiability as
$$
0 = \lim_{h \to 0} \frac{\left|f(x + h) - f(x) - f'(x)(h)\right|}{\left|h\right|}.
\tag{1}
$$
Separately, the difference quotient computations familiar from one variable aren't as convenient when dealing with functions of several variables. For $f(x_{1}, x_{2}) = x_{1}^{2} + x_{2}^{2}$, you'd have
\begin{align*}
0 &= \lim_{h \to 0} \frac{\left|f(x + h) - f(x) - Ah\right|}{\left|h\right|} \\
  &= \lim_{h \to 0} \frac{\left|(x_{1}^{2} + 2x_{1}h_{1} + h_{1}^{2} + x_{2}^{2} + 2x_{2}h_{2} + h_{2}^{2}) - (x_{1}^{2} + x_{2}^{2}) - Ah\right|}{\left|h\right|} \\
  &= \lim_{h \to 0} \frac{\left|(2x_{1}h_{1} + h_{1}^{2} + 2x_{2}h_{2} + h_{2}^{2}) - Ah\right|}{\left|h\right|}.
\end{align*}
But what is $Ah$? Well, okay, it has to be the linear terms in $h$; here, $Ah = 2x_{1}h_{1} + 2x_{2}h_{2}$. But technically, you still have to verify that the preceding limit is zero if $A$ is defined as shown.
In practice, it's easier to "guess" the expression $Ah$ by taking partial derivatives, then to verify that (1) holds for the resulting choice of $A$. For the preceding function, we have
$$
D_{1}f(x_{1}, x_{2}) = 2x_{1},\qquad
D_{2}f(x_{1}, x_{2}) = 2x_{2},
$$
leading us to guess/presume that $Ah = 2x_{1}h_{1} + 2x_{2}h_{2}$. We substitute this in the right-hand side of (1) and check the limit is zero. Recycling the earlier computation, we find
$$
\lim_{h \to 0} \frac{\left|f(x + h) - f(x) - Ah\right|}{\left|h\right|}
  = \dots
  = \lim_{h \to 0} \frac{\left|h_{1}^{2} + h_{2}^{2}\right|}{\left|h\right|}
  = \lim_{h \to 0} \left|h\right| = 0.
$$
The fact that the limit is zero implies simultaneously that $f$ is differentiable at $x$, and that $f'(x)(h) = Ah$.
Similarly, for the function $f(x_{1}, x_{2}) = (x_{1} + x_{2})^{2}$, you "guess" that $f'(x)(h) = 2(x_{1} + x_{2})(h_{1} + h_{2})$. Substituting into (1) and using your computations gives
$$
0 \leq \lim_{h \to 0} \frac{\left|f(x + h) - f(x) - Ah\right|}{\left|h\right|}
  = \dots
  = \lim_{h \to 0} \frac{\left|h_{1}h_{2}\right|}{\left|h\right|}
  \leq \lim_{h \to 0} \left|h_{2}\right| = 0.
$$
(The second inequality comes from the elementary inequality $\left|h_{1}\right|/\left|h\right| \leq 1$ for all $h \neq 0$.)
