Differentiation that involves determinant and vectors How can i calculate following?  
$\frac{d}{dR'} {(|\vec{R}-\vec{R'}|)^{-1}}$  
I tried this but I'm not sure this a valid method:  


*

*let $\vec{R}-\vec{R'} = \vec{u}$, then  

*$-\vec{dR'}= \vec{du}$  


following chain rule we get:
 3. $\frac{d}{dR'} = \frac{d}{du} \times \frac{du}{dR'} $  
it follows from (2), we have:  


*$du = -dR' => \frac{du}{dR'} = -1$ 


so we get
$-u^{-2} \times -1 = \frac{1}{u^{2}} $
 A: You are trying to treat 3d vector variables as scalar variables in a couple of places there, and that doesn't work.
First, as Alex S alludes to, the concept of $\frac{df}{d\vec v}$, where $f(\vec v)$ is a vector-valued function has to be interpreted as the gradient. As such, the differential notation is highly misleading, so it is generally expressed as $\nabla f$ or sometimes $D_{\vec v}f$, but not as $\frac{df}{d\vec v}$. 
Second, as $\vec u$ and $\vec R'$ are both vectors, $\frac {d\vec u}{d\vec R'}$ represents the derivative of a function from $\Bbb r^3$ to $\Bbb r^3$. I suspect you have yet to encounter such derivatives, but if you have, then you should know that they are $3\times 3$ matrices, not scalar values. Now in this case it is just $-I$, so you were not that far off, but you should be clear on the difference. As above, the differential-ratio notation is not preferred. This one is most commonly denoted by $D_{\vec R'}\vec u$
The chain-rule does indeed apply here, even though the derivatives are of different forms. So $$D_{\vec R'}|\vec u|^{-1} = D_{\vec R'}\vec uD_{\vec u}|\vec u|^{-1}$$.
Third is that $|\vec u| = \sqrt{\vec u \cdot \vec u}$, where $\cdot$ is the inner product, so define $f(x) = 1/\sqrt x$, then $|\vec u|^{-1} = f(\vec u \cdot \vec u)$, which by an application of the multi-variate chain rule again gives us $$D_{\vec u}|\vec u|^{-1} = f'(\vec u \cdot \vec u)D_{\vec u}(\vec u \cdot \vec u)$$ Now $f'(x) = -\frac12x^{-3/2}$, and you can see from coordinate calculation that $D_{\vec u}(\vec u \cdot \vec u) = 2\vec u$.
So if you put these pieces together, you can see what the derivative really is: 
$$\begin{align}D_{\vec R'}|\vec u|^{-1} &= D_{\vec R'}\vec uD_{\vec u}|\vec u|^{-1}\\&=(-I)\left(f'(\vec u \cdot \vec u)D_{\vec u}(\vec u \cdot \vec u)\right)\\&=-\left(-\frac12(\vec u \cdot \vec u)^{-3/2}\right)(2\vec u)\\&=\frac{\vec u}{|\vec u|^3}\\&=\frac{\vec R - \vec R'}{|\vec R - \vec R'|^3}\end{align}$$
