1) You can apply the definition. Let $\mu_1,\mu_2,\mu_3$ be such that
$$0=\mu_1(v_1+v_2)+\mu_2(av_1+v_3)+\mu_3(bv_1-v_3)=(\mu_1+a\mu_2+b\mu_3)v_1+\mu_1v_2+ (\mu_2-\mu_3)v_3\ \ \ (*)$$
Since $v_1,v_2,v_3$ is a basis, we must have $0= \mu_1=\mu_2 - \mu_3=\mu_1+a\mu_2+b\mu_3$.
Thus $\mu_2 = \mu_3$ and $\mu_2(a+b)=0$. $B'$ is linearly independent if and only if equation $(*)$ imply $0=\mu_1=\mu_2=\mu_3$. It is now clear that it is true if and only if $a \neq -b$. From the above discussion we know that $a \neq -b$ imply $0=\mu_1=\mu_2 = \mu_3$. Now suppose $a = -b$, if $a=0$, then clearly $B'$ is not linearly independent and if $a \neq 0$, choose $\mu_1 = 0,\mu_2 = \mu_3= 1$. Note that it is sufficient to check if $B'$ is linearly independent to know if it is a basis since it contains $3$ vectors.
2) So you have $u = -v_1+2v_2+v_3$ and you want to find $\mu_1,\mu_2,\mu_3$ such that $u = \mu_1v_1'+\mu_2v_2'+\mu_3v_3'$ whith $B'=\{v_1',v_2',v_3'\}$ (then $[u]_{B'}=(\mu_1,\mu_2,\mu_3)^T$). By identification with equation $(*)$ you get the system
$$\left\{\begin{array}{rcl}-1&=&\mu_1+\mu_2+2\mu_3\\2 &=& \mu_1\\ 1&=&\mu_2-\mu_3 \end{array}\right. \iff \left\{\begin{array}{rcl}\mu_1 &=& 2\\ \mu_2&=& -\frac{1}{3}\\ \mu_3&=& -\frac{4}{3}\end{array}\right.$$