$\left \| \cdot \right \|$ is an induced norm. If $\left \| A \right \|<1$, how to show that $I-A$ is nonsingular and ...? The induced norm $\left \| \cdot  \right \|$ is defined for a matrix
$A\in\mathbb{C}^{n\times n}$ as $\left \| A \right \|=\sup_{||x||=1} \left \| Ax \right \|$. If $\left \| A \right \|<1$, show that (1) $I-A$ is nonsingular and (2) $\left \| \left ( I-A \right )^{-1} \right \|\leq \frac{1}{1-\left \| A \right \|}$.
 A: It can be made quite simple. If $\|A\| < 1$ and you assume by contradiction that $(I-A)$ is singular, then there exists $x \neq 0$ such that $Ax=x$ and without loss of generality, you can scale $x$ so that $\|x\|=1$. But then $\|Ax\| = 1$ and therefore $\sup_{\|x\|=1} \|Ax\| \geq 1$, which is a contradiction.
For the second part, by definition $\|(I-A)^{-1}\| = \sup_{\|y\|=1} \|(I-A)^{-1}y\| = \sup_{\|y\|=1} \|x_y\|$ where $x_y := (I-A)^{-1}y$, i.e., $y = (I-A) x_y$. Now
$$
1 = \|y\| = \|(I-A) x_y\| = \|x_y - A x_y\| \geq \|x_y\| - \|A x_y\|
\geq \|x_y\| - \|A\| \|x_y\| = (1 - \|A\|) \|x_y\|.
$$
Finally, $\|x_y\| \leq 1/(1 - \|A\|)$ and so is the $\sup$.
A: I will show it for $I+A$  which doesn't matter at all 
\begin{align*}
 \| (I+A)x\| &= \| x + Ax\|\\
&\geq \|x\|-\|Ax\|\\
&\geq \underbrace{(1-\| A \|)}_{>0 }\| x\|
\end{align*}
And hence the kernel is trivial and so $(I+A)$ is invertible.
For the second use 
$$\|(I+A)^{-1}b\|=\|x\| \leq \frac{1}{1-\| A \|} \|(I+A)x\|
  =\frac{1}{1-\| A \|} \cdot \| b \|$$
A: $I - A$ is nonsingular means that for $x \not= 0$, $(I - A)x \not= 0$. But expanding that, you get just $x - Ax \not= 0$, i.e. $x \not= Ax$.
So $I - A$ is nonsingular precisely when $A$ does not have any fixed points. Now, $\| Ax \| \le \| A \| \| x \|$ (prove this if you don't already know it), so if $\| A \| < 1$ then $\| Ax \| < \| x \|$, so $A$ can't possibly have any fixed points.
Basically, if you have $\| A \| < 1$ then $A$ makes every vector shorter, so $-Ax$ can't make it all the way back to the origin from $x$.
For the second part, I would use (again, if you don't know these you should prove them) $\| AB \| \le \| A \| \| B \|$ and $\|A + B\| \le \|A\| + \|B\|$ (so in particular, set $B = I - A$ and you get $\| I \| - \|A \| \le \| I - A \|$)
A: Theorem: Let $S\in\text{GL}(\mathbb{C}^n)$ and $T\in\text{End}(\mathbb{C}^n)$ be such that $\|T-S\|_\text{op}\|S^{-1}\|<1$. Then, $T\in\text{GL}(\mathbb{C}^n)$.
It suffices to show that $\ker T$ is trivial. To this end we observe that 
$$\begin{aligned}\frac{\|v\|}{\|S^{-1}\|_\text{op}} &=\frac{1}{\|S^{-1}\|_\text{op}}\|S^{-1}(S(v))\|\\ &\leqslant \frac{\|S^{-1}\|_\text{op}}{\|S^{-1}\|_\text{op}}\|S(v)\|\\ &= \|S(v)\|\\ &\leqslant \|(S-T)(v)\|+\|T(v)\|\\ &\leqslant\|S-T\|_\text{op}\|v||+\|T(v)\|\\ &=\|T-S\|_{\text{op}}\|v\|+\|T(v)\|\end{aligned}$$
And thus we obtain that 
$$\left(\frac{1}{\|S^{-1}\|_\text{op}}-\|T-S\|_\text{op}\right)\|v\|\leqslant\|T(v)\|\quad\mathbf{(1)}$$
Thus, if $v\ne0$ then $\|v\|>0$ and by assumption we may then conclude that the left side of $\mathbf{(1)}$ is positive, and so $\|T(v)\|$ is positive. Thus, $T(v)\ne0$ and so $T$ has a trivial kernel. $\blacksquare$
Taking $S=I$, and $T=I-A$ in your case shows that $I-A$, and thus $A-I$ is invertible. 
Do you see how to prove the norm inequality on the inverse of $I-A$?
A: Referring to a previous problem, for $||A||<1$, we have
$$ (I-A)^{-1} = \sum_{n=0}^{\infty}A^n $$ 
$$\implies  ||(I-A)^{-1}|| = ||\sum_{n=0}^{\infty}A^n||\leq \sum_{n=0}^{\infty}||A^n||\leq \sum_{n=0}^{\infty}||A||^n=\frac{1}{1-||A||}  $$
$$ \implies ||(I-A)^{-1}|| \leq \frac{1}{1-||A||}. $$
Note that, we have used the fact that 
$$ ||A^n||\leq ||A||^n .$$ 
