# By using properties of determinants show that

$$\begin{vmatrix}1+a^2-b^2&2ab&-2b\\ 2ab&1-a^2+b^2&2a\\ 2b&-2a&1-a^2-b^2\end{vmatrix}=(1+a^2+b^2)^3$$

I have been trying to solve the above determinant. But unfortunately my answer is always coming as: $$1+3a^2+3a^4+a^6+3a^2b^4+3b^2+4a^2b^2+a^4b^2+b^6+3b^4$$ Please help me to solve this problem.

-
Your 7th and 8th terms have wrong numerical coefficient, that should be $6$ (instead of $4$) and $3$ (instead of $1$). In this way it becomes a perfect cube of a trinomial. –  enzotib Sep 17 '12 at 13:12
The question asks you to use properties of the determinant to solve it. While you can brute-force it, as you can see it becomes complicated. On the other hand, you can add a scalar multiple of one row to another and the determinant will not change. –  process91 Sep 17 '12 at 13:14

$$\begin{vmatrix}1+a^2-b^2&2ab&-2b\\ 2ab&1-a^2+b^2&2a\\ 2b&-2a&1-a^2-b^2\end{vmatrix}$$

$$=\begin{vmatrix}1+a^2-b^2&2ab&-2b\\ 2ab-a(2b)&1-a^2+b^2-a(-2a)&2a-a(1-a^2-b^2)\\ 2b&-2a&1-a^2-b^2\end{vmatrix}$$ (applying $R'_2=R_2-aR_3$)

$$=\begin{vmatrix}1+a^2-b^2&2ab&-2b\\ 0&1+a^2+b^2&a(1+a^2+b^2)\\ 2b&-2a&1-a^2-b^2\end{vmatrix}$$

$$=(1+a^2+b^2)\begin{vmatrix}1+a^2-b^2&2ab&-2b\\ 0&1&a\\ 2b&-2a&1-a^2-b^2\end{vmatrix}$$

$$=(1+a^2+b^2)\begin{vmatrix}1+a^2-b^2+b(2b)&2ab+b(-2a)&-2b+b(1-a^2+b^2)\\ 0&1&a\\ 2b&-2a&1-a^2-b^2\end{vmatrix}$$ (applying $R'_1=R_1+bR_3$)

$$=(1+a^2+b^2)\begin{vmatrix}1+a^2+b^2&0&-b(1+a^2+b^2)\\ 0&1&a\\ 2b&-2a&1-a^2-b^2\end{vmatrix}$$

$$=(1+a^2+b^2)^2\begin{vmatrix}1&0&-b\\ 0&1&a\\ 2b&-2a&1-a^2-b^2\end{vmatrix}$$

$=(1+a^2+b^2)^2$ $\begin{vmatrix}1&0&-b\\ 0&1&a\\2b+2a(0)-2b(1)&-2a+2a(1)-2b(0)&1-a^2-b^2+2a(a)-2b(-b)\end{vmatrix}$

(applying $R'_3=R_3+2aR_2-2bR_1$)

$=(1+a^2+b^2)^2$ $\begin{vmatrix}1&0&-b\\0&1&a\\0&0&1+a^2+b^2\end{vmatrix}$

$=(1+a^2+b^2)^3$

-
Thank you very much –  Drownpc Sep 17 '12 at 15:18
Nice to hear that, you are most welcome. –  lab bhattacharjee Sep 17 '12 at 15:32

Assuming $a,b\in\mathbb{R}$ (and if not, proving it for real $a,b$ should allow you it to extend it to complex $a,b$), each row vector has the norm $(1+a^2+b^2)$ and the row vectors are all orthogonal so the matrix divided by $(1+a^2+b^2)$ is an orthogonal matrix and the absolute value of its determinant is 1 which immediately implies your result.

-