Reducing the System of linear equations \begin{align*}
x+2y-3z&=4 \\
3x-y+5z&=2 \\
4x+y+(k^2-14)z&=k+2
\end{align*}
I started doing the matrix of the system:
$$ \begin{pmatrix} 1 & 2 & -3 & 4 \\ 3 & -1 & 5 & 2 \\ 4 & 1 & k^2-14 & k+2  \end{pmatrix} $$
But I still don't know how can I reduce this, could you explain me what steps I should follow?
Which values of $k$ make the system inconsistent, determined, undetermined?
 A: Let our augmented matrix be $$A=\begin{pmatrix} 1 & 2 & -3 & 4 \\ 3 & -1 & 5 & 2 \\ 4 & 1 & k^2-14 & k+2  \end{pmatrix}$$ as you pointed out. So, if we do $$-3R_1+R_2\to R_2~(I),~~~ -4R_1+R_3\to R_3~(II),~~~R_2-R_3\to R_3~(III)$$ then we get $$A\to B=\begin{pmatrix} 1 & 2 & -3 & 4 \\ 0 & -7 & 14 & -10 \\ 0 & 0 & 16-k^2 & -k+4  \end{pmatrix}$$ Now think about 


*

*$k=4$ which makes your system consistent with infinitely many solutions,

*$k=-4$ which makes your system inconsistent,
and...
A: Hint
For sure, the most elegant manner would consist in a solution based on matrix calculations, noting that, as pointed out by Eric Towers, $k$ is just a number.  
There is also a more basic solution which consists in variable elimination. From the first equation, you can extract $x$ as a function of $y$ and $z$. Plugging the so obtained  expression of $x$ into the second equation gives you an expression in which only appear $y$ and $z$; then, you can extract $y$ as a function of $z$. Now, plug all previous results into the third equation. After simplifications, you arrive to $$(k-4) ((k+4) z-1)=0$$ from which you can observe what B.S. wrote. 
If you are not in these critical situations, then the solution for $z$ is immediate and going backwards, you find the corresponding $x$ and $y$.
A: Let us name the $3$ by  $3$ matrix of the coefficients as $A$ and the $3$ by $1$ matrix of the constants as $B$. And the matrix that you have provided becomes augmentation of two, name it $C = ([A|B])$
The system is said to be consistent, if we can find at least one solution, otherwise we say it inconsistent. To get an idea of consistency, find the determinant of $A$


*

*If $det (A) \neq 0$ , you have a unique solution and is consistent.

*If $det (A) = 0$ , it can have either no solution (which is inconsistent) or infinitely many solutions (which is consistent).


To proceed further with $det (A) = 0$ condition:


*

*If $Rank(A) \neq Rank(C)$, you have no solution.

*If $Rank(A) = Rank(C) < n$ (the size of $A$), you have infinitely many solutions. (If $Rank(A) = Rank(C) = n$,  you have a unique solution)
You need to reduce the matrix so that you can find the Rank easily (otherwise, in my opinion, reducing the matrices is not going to help). The transformations you need to perform are : 


*

*$R_2 = R_2-3R_1$, it would make $A[2][1]$ as $0$.

*$R_3 = R_3 - 4R_1$, it would make $A[3][1]$ as $0$.

*Continuing so, you should reduce the given matrix to an Upper Triangular matrix (reducing to Upper Triangular form will help you to find the Rank of the matrix)

A: This is not very different from other answers. And I think that the approach using Gaussian elimination is more systematic. I wanted to show how this can be done with a little of guesswork.

If you notice that the sum of the first two equations is $4x+y+2z=6$, then you see that every solution of your system also fulfills the equations
\begin{align*}
4x+y+2z&=6 \\
4x+y+(k^2-14)z&=k+2
\end{align*}
Now if you subtract these equations you get:
$$(k^2-16)z=k-4$$
which is the same as $(k-4)(k+4)z=(k-4)$.
If $\boxed{k=4}$, then this last equation is $0=0$ and you system is equivalent to the system
\begin{align*}
x+2y-3z&=4 \\
3x-y+5z&=2 
\end{align*}
If $\boxed{k=-4}$, then the this equation is $0=-8$, and the system does not have solution.
If $\boxed{k\ne\pm4}$, then the original system is equivalent to the following system of equations:
\begin{align*}
x+2y-3z&=4 \\
3x-y+5z&=2 \\
(k^2-16)z&=k-4
\end{align*}
Since $k^2-16\ne0$, the variable $z$ is uniquely determined by the last equation. Since the matrix $\begin{pmatrix}1&2\\3&-1\end{pmatrix}$ is regular, the values of $x$ and $y$ are the also uniquely determined.
