Finding the sign of a dot product If $|\vec{a}|=2$,  $|\vec{b}|=5$  and $|\vec{a}\times\vec{b}|=8$,
find $(\vec{a }\cdot\vec{b})$
Using $|\vec{a}\times\vec b|^2 + (\vec{a }\cdot\vec{b})^2=|\vec{a}|^2|\vec{b}|^2$
we get two answers +6 and -6. Are both of them correct or should we take the positive value?
 A: The fact that $\vec{a}$ and $\vec{b}$ are vectors is really a red herring.  With the  information given, the equation is $64+ x^2= 100$ so $x^2= 36$ which has roots x= 6 and x= -6.  There is no reason why the dot product of two vectors cannot be negative so both are solutions.
A: If you have vectors $\vec a$ and $\vec b$ that satisfy the given conditions, then replacing $\vec a$ by its inverse will flip the sign of $\vec a\cdot \vec b$, but leave the givens unchanged.
So you can't hope to find the sign of $\vec a\cdot \vec b$ based on those given values.
A: This question is equal to say:  

One parallelogram has sides with lengths $2$ and $5$, and area $8$, what is the angle between them?  

And it's clear that we have two answer:

A: Both of them are correct since $$\vec|a\times\vec b| = 8 \implies ||a||b|\sin\theta| = 8 \implies |sin \theta| = \frac{4}{5}$$
So $$\sin \theta = \pm \frac{4}{5} \implies \cos \theta = \pm \frac{3}{5}$$
Thus,$$(\vec a \cdot \vec b) = |a||b|\cos\theta = \pm 6$$
A: You said this:
$$|\vec{a}\times\vec b|^2 + (\vec{a }\cdot\vec{b})^2=|\vec{a}|^2|\vec{b}|^2$$
But you did omit intermediate steps to reach the conclusion:
$|\vec{a}\times\vec b|^2 + (\vec{a }\cdot\vec{b})^2=|\vec{a}|^2|\vec{b}|^2\sin(\alpha)^2+|\vec{a}|^2|\vec{b}|^2\cos(\alpha)^2=|\vec{a}|^2|\vec{b}|^2(\sin(\alpha)^2+\cos(\alpha)^2)=|\vec{a}|^2|\vec{b}|^2$
The key here is the 1-sized circle equation relating sin and cos:
$$(\sin(\alpha)^2+\cos(\alpha)^2) = 1$$
So: Provided you use always the same angle (i.e. the same order between vectors a and b), that will be true due to the identity. If you have two different values, it is because you can choose to multiply a with b, or b with a (getting $\alpha = |\alpha|$ or $\alpha = -|\alpha|$). You must take that into account beforehand: $\alpha$ will be determined by the operands order, and will determine the sign of both products operations. Finally: Yes, both values will be valid depending on the quadrant of $\alpha$.
