The sum of angles in a triangle is greater than 180? So basically the question goes:
Let $ABC$ be a triangle and that $\cos A=-\frac 3 5$ and $\cos B=\frac 5 {13}$, find $\cos C$. 
What I did was I get the value of $A$ and $B$ by calculating the arccosine of $-3/5$ and $5/13$ and the sum of both of these angles was already greater than $180$, how can I calculate $\cos C$?
 A: I suspect you're meant to calculate $\cos C$ by using the cosine of sums rule, not by computing inverse cosines.

With that in mind, and on the off chance that the minus sign on $-3/5$ is an error, and the cosine of angle $A$ is really meant to be $3/5$, I'll point out
$$
\arccos 3/5 \approx 53.13 \\
\arccos 5/13 \approx 67.38
$$
which leaves $C \approx 180-53.13-67.38 = 59.49$.  And indeed, we see that (assuming the minus sign is spurious)
\begin{align}
\cos(A+B) & = \cos A \cos B - \sin A \sin B \\
          & = \frac{3}{5} \times \frac{5}{13} - \frac{4}{5} \times \frac{12}{13} \\
          & = \frac{15}{65} - \frac{48}{65} = -\frac{33}{65}
\end{align}
That would mean that $\cos C = 33/65$ (because $A+B+C = 180$), and sure enough,
$$
\arccos 33/65 \approx 59.49
$$
A: Hint.-$$\cos A=-\dfrac 3 5\Rightarrow A\approx(90+36.8698975)^{\circ}=126^{\circ}.8698975\\\cos B=\frac 5 {13}\Rightarrow B\approx 67^{\circ}.38013505$$
It follows $$A+B\gt 180^{\circ}$$ so $ABC$ is not a triangle.
A: You need to pay attention to the conditions of the problem, not just compute mechanically.
First, the fact that the angles belong to a triangle tells us that none of them is up to $180°$. Also, that $\cos A<0$ tells us that the angle at $A$ is obtuse. Similarly, the angle at $B$ is acute. This immediately tells us that we expect $\cos C$ to be positive, since otherwise $\hat C$ would be obtuse, which contradicts the condition $A+B+C=180°$.
We can now begin the computation with these facts in mind. You can use the relationships $$\tan^2 x=\frac{1} {\cos^2 x}-1$$ and $$\tan (A+B+C)=\frac{\tan A+ \tan B +\tan C -\tan A\tan B\tan C}{1-\tan A\tan B-\tan B\tan C - \tan C\tan A}=\tan 180°=0\implies \tan A+ \tan B +\tan C -\tan A\tan B\tan C=0$$ to get $\tan A=-4/3$ and $\tan B=12/5$, which when you substitute in the above should give you $\tan C$, which you can put back in the first relationship to give you $\cos C$(note, take the positive square root here).
My own computation (which may be mistaken) tells me that $$\cos C=63/65>0,$$ as expected.
PS. Looking back at my work, I see that $\tan C=-16/63<0$, contrary to expectation (since $C$ must be acute) so that the angles don't belong to a triangle as claimed. Nevertheless, the $\text{cosine}$ of the angle $C$ is a valid one since the trig relations used are valid for all angles.
