Are limits commutative? Generally speaking, is the following true:
$$\lim_{x\to a}f'(x)=\lim_{x\to a}\left(\lim_{h\to 0}\frac{f(x+h)-f(x)}{h}\right)=\lim_{h\to 0}\left(\lim_{x\to a}\frac{f(x+h)-f(x)}{h}\right)$$
 A: Your statement boils down to the continuity of $f'.$ 
However, the derivative, even if it exists for any point $x\in\mathbb{R},$ does not have to be continuous. A classic counterexample is $f(x)=x^2\sin(1/x)$ for $x\neq 0,$ and $f(0)=0.$ 
A: This is not an answer to the question, but I think worth recording in a less ephemeral way than comments. It is based on an observation by @KaviRamaMurthy.
Suppose $f$ is differentiable in a neighbourhood of $a$ and
$\lim_{x \to a} f'(x)$ exists, then this must equal $f'(a)$ as a
result of Darboux's theorem.
Note that the only additional assumption to the question is that $f$
is differentiable at $a$, in particular, no continuity is assumed.
To emphasise, if the two limits in the question exist and $f$
is differentiable at $a$ then the limits are equal.
Aside:
Suppose $\lim_{x \to a} f'(x) = g \neq f'(a)$. Let $\epsilon= { 1\over 2} |g-f'(a)|$ and choose
$\delta>0$ such that if $0<|x-a| < \delta$ then
$|f'(x)-g| < \epsilon$. Then $f'(B(a,\delta)) \subset \{f'(a)\} \cup B(g,\epsilon)$, note that $f'(a) \notin \overline{B(g,\epsilon)}$.
Pick $x$ such that $0<|x-a|< \delta$. Darboux's theorem states that $(f'(x),f'(a)) \subset f'(B(a,\delta))$ which is a contradiction.
A: If the limit of derivative exists at a point $a$, then function can be approximated by a line at points really close to left of $a$, now if the limit of derivative exists, then another line with same slope as previous line approximates the points really close to right of $a$. That was about LHS of the equation, now coming to RHS, your first limit means you approach $a$ from any side, and fix an $x$ really close to $a$, with second limit you are talking about derivative at that $x$, which may not exist, but as you get closer to $a$,    the derivative is defined at those points close to $a$, because your LHS(limit of derivative exists), meaning derivative exists around a neighborhood of $a$. 
As for examples for equality,you can have $y=x$, at non zero $x$, and make this function discontinuous at zero.
