Find the triangular matrix and determinant. I have a 4x4 matrix and I want to find the triangular matrix (lower half entries are zero).
$$A=
\begin{bmatrix}
2 & -8 & 6 & 8\\
3 & -9 & 5 & 10\\
-3 & 0 & 1 & -2\\
1 & -4 & 0 & 6
\end{bmatrix}
$$
Here are the elementary row operations I performed to get it into triangular form.
row swap rows 1 and row 4
$r_2 - 3\cdot r_1$ replacing $r_2$
$r_3 + 3\cdot r_1$ replacing $r_3$
$r_4 - 2\cdot r_1$ replacing $r_4$
I get this matrix
$$A= -
\begin{bmatrix}
1 & -4 & 0 & 6\\
0 & 3 & 5 & -8\\
0 & -12 & 1 & 16\\
0 & 0 & 6 & -4
\end{bmatrix}
$$
I then did $4\cdot r_2 + r_3$ to replace $r_3$ and got
$$A= -
\begin{bmatrix}
1 & -4 & 0 & 6\\
0 & 3 & 5 & -8\\
0 & 0 & 21 & -16\\
0 & 0 & 6 & -4
\end{bmatrix}
$$
I then did $-21\cdot r_4 + 6\cdot r_3$ to replace $r_4$ and got
$$A= -
\begin{bmatrix}
1 & -4 & 0 & 6\\
0 & 3 & 5 & -8\\
0 & 0 & 21 & -16\\
0 & 0 & 0 & -12
\end{bmatrix}
$$
I am not sure if I did this correctly but the determinant of the matrix should be -36. When I multiply the diagonal entries it isn't -36. I can't figure out what I am doing wrong.
 A: 
"I then did -21*row 4 + 6*row 3 to replace row 4 and got"

This is a determinant altering operation and not an elementary operation.
Don't write that $A$ equals something which isn't $A$.
Picking up where you errored and using the same idea you had one gets:
$$\begin{align} \begin{bmatrix}
1 & -4 & 0 & 6\\
0 & 3 & 5 & -8\\
0 & 0 & 21 & -16\\
0 & 0 & 6 & -4
\end{bmatrix}&\leadsto \begin{bmatrix}
1 & -4 & 0 & 6\\
0 & 3 & 5 & -8\\
0 & 0 & 6\cdot 21 & -6\cdot 16\\
0 & 0 & -21\cdot 6 & (-21)\cdot (-4)
\end{bmatrix}\\
&\leadsto \begin{bmatrix}
1 & -4 & 0 & 6\\
0 & 3 & 5 & -8\\
0 & 0 & 6\cdot 21 & -16\\
0 & 0 & 0 & -12
\end{bmatrix}_.\end{align}$$
Making the proper compensation yields
$$\det(A)=-\dfrac{1\cdot 3\cdot (6\cdot 21)\cdot (-12)}{-21\cdot 6}=-36.$$
A: "I then did −21⋅r4+6⋅r3 to replace r4 and got"..
Whenever we are doing Row operations on a particular row, whatever the co-efficients that we are multiplying on the same row should be taken as a divisor with sign outside of determinant.
For example,
You have a matrix A and it's determinant is |A|
If we are doing the below operation,
R3 -> 3 R2 - 5R3
The operation shout be processed by taking (-1/5) outside.
Concept is, we are indirectly multiplying the Row 3 by (-5) through this operation. We no need to worry about the multiplier 3 with R2, as it will not affect the determinant value(we are altering the Row 3, hence remaining Row's co-efficients will not affect |A|).
In your case, Till the final step, the Row operations didn't have co-efficients for the particular Row alterations.
In final step you did,
R4 -> -21 R4 + 6 R3
So, you should take (-1/21) outside. 
After taking outside, the determinant calculation will be like:
|A| = - (-1/21) (1) (3) (21) (-12) = -36
