Determining what values in a system can cause infinite/unique/no solutions. I just did this exercise. I got the right answer except for one minor detail according to my book. I can see the reason, but I'm not sure what sort of logic did the book use to determine that. You can skip to the bottom of the question.

$$\begin{matrix}
px_1&+&2x_2&+&3x_3&=&2\\
px_1&+&px_2&+&(p+1)x_3&=&p\\
px_1&+&px_2&+&(2p-2)x_3&=&2p-2
\end{matrix}$$

For what value or values of $p$ does the system have infinite
  solutions, unique solution, and none at all?

$$\begin{bmatrix}
p&2&3\\
p&p&p+1\\
p&p&2p-2
\end{bmatrix}
\begin{bmatrix}
2\\
p\\
2p-2
\end{bmatrix}$$
$$-r_1+r_2 \ \ , \ \ -r_1+r_3$$
$$\begin{bmatrix}
p&2&3\\
0&p-2&p-2\\
0&p-2&2p-5
\end{bmatrix}
\begin{bmatrix}
2\\
p-2\\
2p-4
\end{bmatrix}$$
$$-r_2+r_3$$
$$\begin{bmatrix}
p&2&3\\
0&p-2&p-2\\
0&0&p-3
\end{bmatrix}
\begin{bmatrix}
2\\
p-2\\
p-2
\end{bmatrix}$$

CASE A - If $p = 2$:
$$\begin{bmatrix}
2&2&3\\
0&0&0\\
0&0&-1
\end{bmatrix}
\begin{bmatrix}
2\\
0\\
0
\end{bmatrix}$$
We have infinite solutions depending on one variable.

CASE B - If $p \not = 2$:
$$\begin{bmatrix}
p&2&3\\
0&p-2&p-2\\
0&0&p-3
\end{bmatrix}
\begin{bmatrix}
2\\
p-2\\
p-2
\end{bmatrix}$$
$$\frac{1}{p}r_1 \ \ , \ \ \frac{1}{p-2}r_2$$
$$\begin{bmatrix}
1&2/p&3/p\\
0&1&1\\
0&0&p-3
\end{bmatrix}
\begin{bmatrix}
2/p\\
1\\
p-2
\end{bmatrix}$$
CASE B.1 - If $p = 3$:
$$\begin{bmatrix}
1&2/3&3/3\\
0&1&1\\
0&0&0
\end{bmatrix}
\begin{bmatrix}
2/3\\
1\\
1
\end{bmatrix}$$
We have an inconsistent system. There is no solution for $p = 3$.
CASE B.2 - If $p \not = 3$:
$$\begin{bmatrix}
1&2/p&3/p\\
0&1&1\\
0&0&p-3
\end{bmatrix}
\begin{bmatrix}
2/p\\
1\\
p-2
\end{bmatrix}$$
$$\frac{1}{p-3}r_3$$
$$\begin{bmatrix}
1&2/p&3/p\\
0&1&1\\
0&0&1
\end{bmatrix}
\begin{bmatrix}
2/p\\
1\\
(p-2)/(p-3)
\end{bmatrix}$$
We have an unique solution.

ANSWER:


*

*For infinite solutions, we need $p = 2$.

*For unique solution, we need $p \in \mathbb{R} - \{ 2, 3\}$.

*For no solution, we need $p = 3$.



THE BOOK'S ANSWER:
Exactly like my answer above, except that it says, additionally:

For $p = 0$, there is no solution.

Hm. Let's see why. If we replace $p = 0$:
$$\begin{bmatrix}
0&2&3\\
0&0&1\\
0&0&-2
\end{bmatrix}
\begin{bmatrix}
2\\
0\\
-2
\end{bmatrix}$$
And of course, now I see why. Since the last row is reduced to
$$\begin{bmatrix}
0&2&3\\
0&0&1\\
0&0&1
\end{bmatrix}
\begin{bmatrix}
2\\
0\\
1
\end{bmatrix}$$
This becomes inconsistent.

It would have never occurred to me that $p = 0$ was a case worth branching off. So my question is: what reasoning did the book use to decide that $p = 0$ was worth branching off?
 A: Let
$$A=\begin{bmatrix}
p&2&3\\
p&p&p+1\\
p&p&2p-2
\end{bmatrix}$$
Then
$$\det A=p(p-2)(p-3)$$
So you should check each value of $p$ to see if the equations become over-determined, under-determined or just right.
If it is over-determined, then usually no solution.
If it is under-determined, then usually infinite solutions.
If it is just right, then usually one solution.
A: Regarding your question 

what reasoning did the book use to decide that $p = 0$ was worth
  branching off?

One approach to this kind of problems is to find any value of $p$ that would cause one or more of the following conditions to occur:


*

*A redundant row (2 or more columns become equal) - This is the case when you have more equations than variables. 

*A redundant column (2 or more rows become equal) - This is the case when you have more variables than equations.

*A row that is entirely zero

*A column that is entirely zero

*The Right hand-side becomes all zero - This is the case when the solution vector must all be zero.
Values of $p$ that make non-of the above possible are the values where the system has a non-zero unique solution.
When $p=0$, condition (4) arises and is observable immediately.
A: Hint: The determinant of the square matrix factors as $p(p-2)(p-3)$, as can be seen from the common factor in column 1, row 2 - row 3 and column 1 - column 2.
