how to find $(I + uv^T)^{-1}$ Let $u, v \in \mathbb{R}^N, v^Tu \neq -1$. Then I know that $I +uv^T \in \mathbb{R}^{N \times N}$ is invertible and I can verify that $$(I + uv^T)^{-1} = I - \frac{uv^T}{1+v^Tu}.$$ But I am not able to derive that inverse on my own. How to find it actually? That is if am given $A =  I +uv^T$ and I am asked to find $A^{-1}$, how to get the answer?
 A: If you are just looking for a way to come up with a guess, you could try starting with $(1+u^Tv)^{-1} = 1 - u^Tv + (u^Tv)^2 + \cdots = 1 - (u^Tv - (u^Tv)^2 + (u^Tv)^3 - \cdots)$. We could factor a $u^Tv$ on the left of the bracketed term, or the right, but that will not give anything useful: instead take a $u^T$ out on the left and a $v$ on the right to get
$$1 - u(1-v^Tu + (v^Tu)^2 - \cdots)v^T $$
which for small $v^Tu$ is
$$ 1 - \frac{uv^T}{1+v^Tu}$$
These manipulations are not legitimate in general, but they do at least suggest that this might be the correct answer, and you can then check that it really is correct.
A: If you guess that the inverse has a similar form,
$$(I+uv^T)^{-1}=\alpha I+\beta uv^T\ ,$$
then multiplying out and equating coefficients will solve the problem. 
A: This is essentially the same as Matthew's answer, but written the right way around.
Recall the geometric series:
$$ 1+q+q^2 + q^3 + … =\sum_{k=0}^∞ q^k =  \frac{1}{1-q}\qquad\text{for $|q|<1$}$$
This series works the same way for square matrices, but is then often called Neumann Series instead.
$$  + A + A^2 + A^3 + … = \sum_{k=0}^∞ A^k =  (-A)^{-1}\qquad\text{for $ρ(A)<1$}
$$
Here $ρ(A) = \max\{|λ|∣ \text{λ is an eigenvalue of $A$}\}$ is the spectral radius of $A$. Note that $ρ(A)<‖A‖$ for any induced matrix norm.
That's basically it.

Now, let's apply this formula to $+uv^⊤$, assuming $‖uv^⊤‖<1$. Note that
$$(-uv^⊤)^k = -(uv^⊤)(uv^⊤)…(uv^⊤) = -u(v^⊤u)(v^⊤u)…(v^⊤u)v^⊤ = -(-v^⊤u)^{k-1}uv^⊤$$
Thus
$$(+uv^⊤)^{-1} = ∑_{k=0}^{∞} (-uv^⊤)^k =   - \Big( ∑_{k=1}^{∞} (-v^⊤u)^{k-1}\Big)uv^⊤  =  - \frac{uv^⊤}{1-v^⊤u}$$
Note that the last step works because $‖uv^⊤‖ <1 ⟹ |v^⊤ u| <1$. After the inverse is found in this case it is easy to check that the formula works, even when $‖u^⊤v‖≥1$, except of course when $v^⊤u = -1$, in which case $+uv^⊤$ is not invertible.
In the case where you have $A+uv^⊤$ with $A$ invertible, you simply apply the formula on
$$ (A+uv^⊤)^{-1} = \Big(A(+A^{-1}uv^⊤)\Big)^{-1} = (+A^{-1}uv^⊤)^{-1}A^{-1}$$
