Differentiate $y=\frac 1 x$ from first principles Here's my attempt:
$$\frac{dy} {dx}=\lim_{\delta x\to 0}\frac{(x+\delta x)^{-1}-x^{-1}}{\delta x} $$
$$\frac{dy} {dx}=\lim_{\delta x\to 0}\frac{\frac1{\delta x}(1+x)^{-1}-x^{-1}}{\delta x} $$
Then I used the binomial expansion (I recognize that this is unconventional but I have done one like this before and I would like to be able to do it again - I just can't get the right result) Please can you give me a solution that uses the binomial expansion if it is valid. If there is a quicker way, please also explain that.
$$\frac{dy} {dx}=\lim_{\delta x\to 0}\frac{(\frac 1 {\delta x}-\frac x {\delta x^2} +\frac {x^2} {\delta x^3}+...)-x^{-1}}{\delta x} $$
This is the point at which I have become stuck
 A: I'm not using binomial expansion, but the following way seems much simpler.
We wish to find
$$\frac{df}{dx} = \lim_{h\to 0}\frac{f(x+h)-f(x)}{h},$$
where $f(x) = 1/x$. Use the following steps:
\begin{align}
\frac{\frac{1}{x+h}-\frac{1}{x}}{h} &= \frac{\frac{x}{(x+h)x}-\frac{x+h}{(x+h)x}}{h}\\
&=\frac{\frac{-h}{x(x+h)}}{h}\\
&=\frac{-1}{x(x+h)}\\
&=\frac{-1}{x^2+xh}.
\end{align}
A: Your algebra is the problem. Namely the first equality here.
$$\begin{align}
\frac{1}{x+\delta x}-\frac{1}{x}&=\frac{1}{x}\left(\frac{1}{1+\frac{\delta x}{x}}\right)-\frac{1}{x}
\\&=\frac{1}{x}\left(\frac{1}{1+\frac{\delta x}{x}}-1\right)
\\&=\frac{1}{x}\left(1-\frac{\delta x}{x}+\mathcal{O}\left(\left(\frac{\delta x}{x}\right)^2\right)-1\right)
\\&=\frac{1}{x}\left(-\frac{\delta x}{x}+\mathcal{O}\left(\left(\frac{\delta x}{x}\right)^2\right)\right)
\end{align}$$
Thence
$$\frac{\frac{1}{x+\delta x}-\frac{1}{x}}{\delta x}=-\frac{1}{x^2}+\left(\text{terms of the form } \pm\frac{(\delta x)^{n-1}}{x^{n+1}}\right).$$
However I concur with everyone else --- why make it more complicated?
A: Your approach is not unheard of. In the case of taking a derivative with respect to a function of a real variable, differentiating $f(x) = 1/x$ is fairly straightforward by using ordinary algebra. As follows:
$$f'(x) = \lim_{h\to 0} \frac{\frac{1}{x+h} - \frac1x}{h} = \lim_{h\to 0} \frac{x-(x+h)}{(x+h)xh} = \lim_{h\to 0} \frac{-1}{x(x+h)} = -\frac{1}{x^2}.$$
This is straightforward. However, when taking the derivative with of $f(A)=A^{-1}$, where $A$ is an invertible operator, this process needs to be refined. The problem arises since the space of operators is not commutative (i.e. matrix multiplication does not commute).
The derivative has a slightly different definition in this case, a function is said to be differentiable at $A$ in this setting if for any operator $B$ there exists an operator $C$ such that $$\frac{\|f(A+hB)-f(A)-Ch\|}{h} \to 0$$ as $h\to 0$. (I apologize for being imprecise here.)
We will need to find an expression for $(A+hB)^{-1} - A^{-1}$ in order to establish this. This is where the geometric series will come into play. Note that if $\|A\| < 1$ then $(I-A)^{-1} = \sum_{n=0}^\infty A^n$. Also note that $(ST)^{-1} = T^{-1} S^{-1}$. Thus $$A^{-1} A (A+hB)^{-1} - A^{-1} = A^{-1} ((A+hB)A^{-1})^{-1} - A^{-1}$$
$$=A^{-1}(I+hBA^{-1})^{-1}-A^{-1} = A^{-1} \left( \sum_{n=0}^\infty (-1)^n h^{n} (BA^{-1})^n - I \right)$$
$$=-A^{-1}BA^{-1}h + A^{-1}(BA^{-1})^2 h^2 - \cdots$$
Thus $$\frac{\|(A+hB)^{-1} - A^{-1} + A^{-1}BA^{-1}h\|}{h} \to 0$$ as $h\to 0$, and the (directional) derivative of $f$ at $A$ in the direction of $B$ is $-A^{-1}BA^{-1}$. Note that if $B=I$ then this is simply $-A^{-2}$.
