values of a and b so that diagonal matrix given $A = \begin{pmatrix} 1 & a & 0 & 0 \\ 0 & b & 0 & 0  \\ -b & 0 & 1 & b \\ -a + 1 & a & 0 & a \end{pmatrix}$ Which values of a and b s.t. A is diagonalisable?
My attempt:
My idea was to find the eigenvalues of the matrix A, afterwhich I would consider each eigenvalue separately and try to form the corresponding eigenspace, for instance,
I got the eigenvalues: $\lambda = 1,1,a,b$, then for the eigenvalue $\lambda = 1$ we need the eigenspace to have dimension 2, so solving $A\underline{v} = \underline{v}$ I get 4 equations:
$ x_1 + ax_2 = x_1 \implies ax_2 = 0$
$bx_2 = x_2 \implies x_2(1-b) = 0$
$-bx_1 + x_3 + bx_4 = x_3 \implies b(x_4 - x_1) = 0$
$x_1 - ax_1 + ax_2 + ax_4 = x_4 \implies x_1(1-a) = x_4(1-a)$, then I really just looked at the equation to see how I would get 2 free variables and noted that $a \not= 1$ and $b \not= 1$ is required, but I don't really know how to explain it, or if it's even correct. For instance, if $a = 1$ then we'd get the dimension to be 3, so $a \not= 1$, similarly if b = 1 then we'd also get the dimension to be 3 so we need $b \not = 1$ this is what I'm thinking.
From $\lambda = b$ I get another requirement of $a \not = 0$, all others give the same requirements as above, or none so I think we need $a \not = 1$ and $a \not = 0$ and $b \not = 1$
Any help would be appreciated thanks
 A: The matrix looks like it will be in better shape if we permute the basis. Form a new bases $[e_3,e_4,e_1,e_2]$ (where  $[e_1,e_2,e_3,e_4]$ is the old basis), on which the matrix is
$$
A'=
  \begin{pmatrix} 1 & b & -b & 0 \\ 0&a &-a + 1 & a \\  0 & 0& 1 & a \\ 0 & 0 &  0 & b \end{pmatrix}.
$$
Now it is clear that the characteristic polynomial is $(X-1)^2(X-a)(X-b)$ and the matrix is diagonalisable if and only if the minimal polynomial has distinct roots, which is the case when the matrix is annihilated by the product of the distinct linear factors in this product. It is clear that the cases $a=1$ and $b=1$ need some special attention.


*

*If $a=b=1$ then the $A'$ is not diagonalisable (nor is therefore $A$), since it is unitriangular but not the identity; this case will henceforth be excluded.

*If $a=1$ then the matrix must be annihilated by $(X-1)(X-b)$ in order to be diagonalisable. Considering the top-left $2\times2$ submatrix this won't happen unless $b=0$, and in that case the matrix is indeed annihilated by $(X-1)X$, so diagonalisable.

*If $b=1$ then the matrix must be annihilated by $(X-1)(X-a)$ in order to be diagonalisable. Considering the lower-right $2\times2$ submatrix this won't happen unless $a=0$, and in that case the matrix is indeed annihilated by $(X-1)X$, so diagonalisable.

*Evaluating $(X-1)(X-a)$ in $X:=A'$ gives a matrix whose first three columns are $0$ and the last column is a multiple of $b-a$. Thus if $b=a\neq1$, then $A'$ is annihilated by this polynomial and hence diagonalisable. If $1,a,b$ are all distinct, then $(X-1)(X-a)(X-b)$ annihilates $A'$, which is therefore also diagonalisable in this case.


Summarising, $A$ is diagonalisable if and only if either $1\notin\{a,b\}$ or if $\{a,b\}=\{0,1\}$.
Given the conciseness of the conclusion, and the at some point surprising vanishing of entries in the above evaluations, one may suspect that something rather special is going on. Indeed it turns out that the eigenspace for $\lambda=1$ always contains not just the obvious $e_3$ but also $e_1+e_4$, independently of $a$, $b$ (this is very atypical behaviour). Having observed this, it is natural to not just permute the basis, but transform to a basis such as $[e_3,e_1+e_4,e_1,e_2]$ instead. With respect to that basis the matrix becomes
$$
A''=
  \begin{pmatrix} 1 & 0 & -b & 0 \\ 0& 1 &-a + 1 & a \\  0 & 0 & a & 0 \\ 0 & 0 &  0 & b \end{pmatrix},
$$
and the conclusion above is now much more transparently obtained. No doubt the original matrix was obtained (by whoever posed the question) by conjugating this $A''$ (or a matrix very similar to it).
A: This problem is more complex than it seems - you HAVE to consider combinations of values of $a$ and $b$...
Your logic in solving the problem is correct, but there are some problems in determining the eigenspace. One should be careful of row reduction in a situation like this, since for some choices of $a$ and $b$ this could lead to division by zero and similar problems. So the coefficient matrix I get, in solving for the eigenspace associated with $1$ is \begin{equation} \begin{bmatrix} 0 & -a&0&0\\0&(1-b)&0&0\\b&0&0&-b\\-(1-a)&-a&0&(1-a)\end{bmatrix}.\end{equation}
So from this it is immediately apparent that $(0,0,1,0)^T$ is an eigenvector associated with 1, regardless of the choice of $a$ and $b$. Now suppose $a=1$, then the coefficient matrix must change to \begin{equation} \begin{bmatrix} 0 & -1&0&0\\0&(1-b)&0&0\\b&0&0&-b\\0&-1&0&0\end{bmatrix},\end{equation} so that $x_2=0$, BUT if $b=0$ then we can have $x_1$ and $x_4$ independent parameters, and so we will have two additional eigenvectors associated with $1$: $(1,0,0,0)^T$ and $(0,0,0,1)^T$. And since $b=0$ is multiplicity $1$ we then have a diagonalizable matrix. So the combination $a=1$, $b=0$ will produce a diagonalizable matrix. 
Now next I would consider the case $b=1$ - if you follow a similar process you will see that actually $b=1$, $a=0$ also works.
Furthermore, for arbitrary $a$ and $b$ such that $a\neq b$ and $a \neq 1 \neq b$ the matrix is diagonalizable - you can check that in such a case the eigenspace associated with $1$ is always dimension $2$ and since $a$ and $b$ are both multiplicity 1 the matrix is diagonalizable. 
Then you should also consider the case $a=b\neq 1$...I think that should do it?
