Find value of $|(\overrightarrow{a} \times\overrightarrow{b}) \times\overrightarrow{c}|$ 

let $\overrightarrow{a} = 2 \hat {i}+ \hat {j}-2 \hat {k}$ and  $\overrightarrow{b}= \hat {i} + \hat{j}$. $\:$ if $\overrightarrow{c}$ is a vector such that $\overrightarrow{a} \cdot \overrightarrow{c}+2|\overrightarrow{c}|=0$ and $|\overrightarrow{a}-\overrightarrow{c}|= \sqrt{14}$ and angle between $\overrightarrow{a} \times \overrightarrow{b}$ and $\overrightarrow{c}$ is $30 ^\circ$. Then value of $|(\overrightarrow{a} \times\overrightarrow{b}) \times\overrightarrow{c}|$ is ?


My first approach:
$|\overrightarrow{a}-\overrightarrow{c}|=\sqrt{|\overrightarrow{a}|^2+|\overrightarrow{c}|^2-2\overrightarrow{a} \cdot\overrightarrow{c}}= \sqrt{14}$
after substituting value of $\overrightarrow{a} \cdot \overrightarrow{c}=-2|\overrightarrow{c}|$ and $|\overrightarrow{a}|=3$ in above equation. I obtained $9+|\overrightarrow{c}|^2+4|\overrightarrow{c}|=14$
$\implies$ $|\overrightarrow{c}|^2+4|\overrightarrow{c}|-5=0$
$\implies$ $|\overrightarrow{c}|=1$
$\overrightarrow{a} \times \overrightarrow{b}=2 \hat{i}-2\hat{j}+ \hat{k}$
$\implies$ $|\overrightarrow{a} \times \overrightarrow{b}|=3$
So from above two result
$|(\overrightarrow{a} \times \overrightarrow{b}) \times \overrightarrow{c}|= |\overrightarrow{a} \times \overrightarrow{b}||\overrightarrow{c}|sin(30^\circ)=(3)(1) \frac{1}{2}= \frac{3}{2}$.
My second Approach:
$|(\overrightarrow{a} \times\overrightarrow{b}) \times\overrightarrow{c}|=|(\overrightarrow{a} \cdot \overrightarrow{c})\overrightarrow{b}-(\overrightarrow{b} \cdot \overrightarrow{c})\overrightarrow{a}|$
To calculate $\overrightarrow{b} \cdot \overrightarrow{c}$ $\;$I took help of
$[\overrightarrow{a} \; \overrightarrow{b} \; \overrightarrow{c}]^2=\begin{vmatrix} \overrightarrow{a}\cdot \overrightarrow{a} & \overrightarrow{a}\cdot \overrightarrow{b}   &\overrightarrow{a}\cdot \overrightarrow{c}  \\ \overrightarrow{b}\cdot \overrightarrow{a} & \overrightarrow{b}\cdot \overrightarrow{b}   &\overrightarrow{b}\cdot \overrightarrow{c} \\ \overrightarrow{c}\cdot \overrightarrow{a} & \overrightarrow{c}\cdot \overrightarrow{b}   &\overrightarrow{c}\cdot \overrightarrow{c} \end{vmatrix}$
$\overrightarrow{a} \cdot \overrightarrow{a}=9$ & $\overrightarrow{b} \cdot \overrightarrow{b}=2$ $\overrightarrow{c} \cdot \overrightarrow{c}=1$ & $\overrightarrow{a} \cdot \overrightarrow{b}=3$ & $\overrightarrow{a} \cdot \overrightarrow{c}=-2$ and let $\overrightarrow{b} \cdot \overrightarrow{c}=x$
$[\overrightarrow{a} \; \overrightarrow{b} \; \overrightarrow{c}]=(\overrightarrow{a} \times \overrightarrow{b}) \cdot \overrightarrow{c} = |\overrightarrow{a} \times \overrightarrow{b}| |\overrightarrow{c}| cos(\theta)=\frac{3 \sqrt3}{2}$


here $\theta$ = angle between $\overrightarrow{a} \times \overrightarrow{b}$ and $\overrightarrow{c} $


$\frac {27}{4}=\begin{vmatrix} 9 & 3 & -2 \\ 3 & 2 & x \\ -2 & x & 1\end{vmatrix}$
After solving above determinant I obtained a equation which is $36 x^2+48x+23=0$
$\implies$ no real value of $x$ that is no real $\overrightarrow{b} \cdot \overrightarrow{c}$
Doubt: Why am I not getting answer using second method?
 A: The first approach gives you a result for $(\vec a \times \vec b) \times \vec c \ $ as it does not test the compatibility of the given data.
Just simply applying,
$|(\overrightarrow{a} \times\overrightarrow{b}) \times\overrightarrow{c}|=|(\overrightarrow{a} \cdot \overrightarrow{c})\overrightarrow{b}-(\overrightarrow{b} \cdot \overrightarrow{c})\overrightarrow{a}|$
shows that there cannot be vectors $\vec a, \vec b$ and $\vec c$ with given conditions.
We know $|a \times b| = 3, |a| = 3, |b| = \sqrt2$
Say the angle between $\vec b$ and $\vec c$ is $\alpha$.
$\displaystyle \frac{3}{2} \ |c| = | \ (-2 \  |c|) \ (\hat i + \hat j) - \sqrt2 \ |c| \cos\alpha \ (2\hat i + \hat j - 2 \hat k) \ | $
Dividing by $|c|$ on both sides,
$\displaystyle \frac{3}{2} = |-2 \hat i - 2 \hat j - \sqrt2 \cos\alpha \ (2 \hat i + \hat j - 2 \hat k) \ |$
$\dfrac{9}{4} = (2 + 2 \sqrt2 \cos\alpha)^2 + (2+ \sqrt2 \cos\alpha)^2 + (2\sqrt2\cos\alpha)^2$
Simplifying, $ \displaystyle \cos^2\alpha + \frac{2\sqrt2}{3} \cos\alpha + \frac{4}{9} = \frac{1}{8}$
$\displaystyle \left(\cos\alpha + \frac{\sqrt2}{3}\right)^2 = - \frac{7}{72}$
which has no real solution.
A: The simple answer is that, the question is erroneous and $\overrightarrow{c}$ simply does not exist.
Note that from your calculations, $\overrightarrow{a}\cdot \overrightarrow{c}=-2|\overrightarrow{c}|$, and $|\overrightarrow{c}|=1$. Since $|\overrightarrow{a}|=3$, this means that if $\phi$ is the angle between these two vectors, then $\cos \phi=-\frac 23$. So, $\phi\approx 131°$.
Now let us demonstrate that this is impossible.
For this purpose, I shall shift to a different reference frame, as that'll help in calculations as well as intuition. Let us consider the frame where $\overrightarrow{a}=3\hat{i}$, with $\overrightarrow {b}$ lying on the $xz$ plane. In this frame, $\overrightarrow{a}×\overrightarrow{b}=3\hat{j}$. Since angle between $\overrightarrow {c}$ and $3\hat{j}$ would be $30°$ in this frame too, we can imagine $\overrightarrow {c}$ to form a cone of semi-vertical angle $30°$ with the $y$-axis, slant length $1$, and vertex at origin.
What would the maximum possible angle between this cone and the $x$- axis ($3\hat{i}$, specifically) be? It is easy to see that this angle cannot exceed $120°$. If you are having trouble gaining this intuition you can observe that the component of $\overrightarrow {c}$ along the $xz$ plane traces a circle of radius $\frac 12$, and it's vertical component is fixed at $1\cdot \cos(30°)=\frac {\sqrt 3}{2}$. Hence, in general, $\overrightarrow {c}$ in this frame is given by $$\overrightarrow {c}=\frac 12 \cos \theta\hat{i}+\frac {\sqrt 3}{2} \hat {j}+\frac 12\sin\theta\hat{k}$$
Thus, doing a dot product shows that angle cannot exceed $120°$. Since $\phi>120°$, the situation is impossible.
