Cross product in complex vector spaces When inner product is defined in complex vector space, conjugation is performed on one of the vectors. What about is the cross product of two complex 3D vectors?
I suppose that one possible generalization is 
$A\otimes B  \rightarrow  \left ( A\times B \right )^*$  where $\times$ denotes the normal cross product. The conjugation here is to ensure that the result of the cross product is orthogonal to both vectors $A$ and $B$. Is that correct ?
 A: For finding the correct definition to apply, one needs to know whether the scalar product is taken to be anti-linear in its first or its second argument. Assuming the first convention, the relation one would want to preserve for $\vec x=(x_1,x_2,x_3)$ and similarly for $\vec y, \vec z$, that one still has
$$
  (\vec x \times \vec y)\cdot\vec z=
  \left|\begin{matrix}x_1&y_1&z_1\\x_2&y_2&z_2\\x_3&y_3&z_3\\\end{matrix} 
  \right|.
$$
Note that the determinant is linear in all of its columns, so the left hand side needs to be an expression that is linear in the vector that appears directly as a column, which explains that one cannot use $\vec x\cdot(\vec y\times\vec z)$ instead, which is anti-linear in $\vec x$. Now it is easy to see that the coordinates of $\vec x \times \vec y$ should be taken to be the complex conjugates of the expressions in their usual definition, for instance $\overline{x_2y_3-x_3y_2}$ for the first coordinate.
One actually arrives at the same conclusion for a scalar product that is defined to be anti-linear in its second argument. However the identity that leads to this definition is different, namely the one which equates $\vec x\cdot(\vec y\times\vec z)$ to the above determinant. 
A: Yes, this is correct definition. If $v$, $w$ are perpendicular vectors in $\Bbb C^3$ (according to hermitian product) then $v,w,v\times w$ form matrix in $SU_3$. 
We can define complex cross product using octonion multiplication (and vice versa). Let's use Cayley-Dickson formula twice: $$(a+b^\iota)(c+d^\iota)=ac-\bar db+(b\bar c+da)^\iota$$ for quaternions $a,b,c,d$. Next set $a=u\mathbf j, b=v+w\mathbf j, c=x\mathbf j, d=y+z\mathbf j$ for complex numbers $u,v,w,x,y,z$. Then we obtain from above formula $$-u\bar x-v\bar y-\bar w z+(\bar vz-w\bar y)\mathbf j+[w\bar x-\bar uz+(-vx+uy)\mathbf j]^\iota$$
Applying complex conjugation to third complex coordinate we obtain formula for cross product. The first term is hermitian product of the vectors $(u,v,w)$, $(x,y,z)$. $$\begin {bmatrix}u\\ v \\ w  \end{bmatrix}\times \begin {bmatrix}x\\ y \\ z  \end{bmatrix}=\begin {bmatrix}\overline {vz}-\overline{wy}\\ \overline{wx}-\overline{uz} \\ \overline{uy}-\overline{vx}  \end{bmatrix}$$
$SU_3$ is subgroup of octonion automorphism group $G_2$. Any automorphism of octonions can be obtained by fixing unit vector $\mathbf i$ on imaginary sphere $S^6$. It defines complex structure on perpendicular space $R^6$ via multiplication. Now in this complex structure any $SU_3$ element is octonion automorphism. So $G_2$ is fiber bundle $S^6 \times \times SU_3$.
Now going to "vice versa". Let's define octonions as pairs $(a,\mathbf v)$ where $a$ is complex number and $\mathbf v$ vector in $\Bbb C^3$. Then octonion multiplication can be defined as $$(a,\mathbf v)(b,\mathbf w)=(ab-\mathbf {v\cdot w},a\mathbf w+b \mathbf v + \mathbf {v \times w})$$
I hope that above argument with double Cayley-Dickson formula can be used to prove it although I have not done myself this calculation. The reader is urged to do it as an exercise :)
We can extend the definition of cross product to quaternions the same way. Extending it to octonions we need to be more careful. Freudenthal has done this using 3x3 matrices over octonions - so called Jordan algebra. Some kind of "cross product" is present in all exceptional Lie groups $F_4$, $E_6$, $E_7$, $E_8$ as these groups are called by Rosenfeld as automorphism groups of 2-dimensional projective planes over $\Bbb {O,C\otimes O, H \otimes O, O \otimes O}$.
Have I flied away too far from original question ?
A: Physicists tend to use the (nonconjugated) "real formulae" for both dot product and cross products. So physicists' dot product equals $\vec x\cdot \vec y = \vec x^T \vec y$, although their inner product is $\langle\vec x,\vec y\rangle=\vec x^H \vec y$ (or $\left\langle \vec x|\vec y \right\rangle$), where $\vec x^H = \mathrm{conj}(\vec x)^T$ (the transpose with complex-conjugated elements).
Usually physicists write $\vec x^\dagger$ instead of $\vec x^H$, whereas mathematicians may write $\vec x^*$ for $\vec x^H$ (alternatively, for the conjugate of $\vec x$, which is why I use here the Hermitian notation $\vec x^H$, to avoid ambiguity).
Non-conjugated dot and cross products are the standard practice in, e.g., almost all books on electromagnetism. Note also that physicists, modern mathematicians and almost everyone else tend to conjugate the first element in their inner products, as above, except that old-school mathematicians more often the second element, according to Wikipedia, Note 2: https://en.wikipedia.org/wiki/Inner_product_space#Notes
However, the following Wikipedia page claims that linearity in the first coordinate is [still] the prevailing condition in mathematics (not merely old-school), and uses $\left\langle \cdot ,\cdot \right\rangle$ for that and $\left\langle \cdot |\cdot \right\rangle$ for linearity in the second argument: https://en.wikipedia.org/wiki/Riesz_representation_theorem#Mathematics_vs._physics_notations_and_definitions_of_inner_product
In either case, physicists get the following formula:
$$
  (\vec x \times \vec y)\cdot \vec z=
  \left|\begin{matrix}x_1&y_1&z_1\\x_2&y_2&z_2\\x_3&y_3&z_3\\\end{matrix} 
  \right|.
$$
Mathematicians tend to have dot product equal to inner product, i.e., $\vec x\cdot \vec y = \langle\vec x,\vec y\rangle=\vec x^H \vec y$ for modern mathematicians, and $=\vec y^H \vec x$ for old-school mathematicians. (Note the old-school jumping order that makes their dot products linear with respect to their first arguments and conjugate-linear w.r.t. the second ones.)
Therefore, mathematicians tend to define cross$(x,y)=$ conj(physicscross$(x,y))$, i.e.,
$$ \vec x \times \vec y = \left[\begin{matrix}\overline{x_2y_3-x_3y_2}\\ 
 \overline{x_3y_1 - x_1y_3}\\
 \overline{x_1y_2-x_2y_1}\\
\end{matrix} 
  \right]
$$
(the overline symbols denote complex conjugation; remove them for physicists' cross product)
to preserve the formula (if conjugating the 1st argument of dot products)
$$
  (\vec x \times \vec y)\cdot\vec z=
  \left|\begin{matrix}x_1&y_1&z_1\\x_2&y_2&z_2\\x_3&y_3&z_3\\\end{matrix} 
  \right|.
$$
Old-school mathematicians have, instead, $\vec x \cdot (\vec y\times\vec z)=|\cdots|$,
as their dot product (conjugating the 2nd argument) conjugates the latter element, thus canceling the conjugation in the cross product this way.
The physicists' (non-conjugated) formulae can be found, e.g., here (gated): https://www.tandfonline.com/doi/pdf/10.1080/002073999287815
Matlab, Maple and Mathematica
Matlab uses modern mathematicians' formulae for the dot product, i.e., it conjugates the first element: dot$(x,y)=x^Hy$ (link https://se.mathworks.com/help/matlab/ref/dot.html); but it uses the physicists' (non-conjugated) formulae for the cross product: cross$(x,y)$ doesn't involve the conjugation of the output.
Thus, in Matlab, dot(i,1) = -i,
cross([0,i,0],[0,0,1]) = (+i, 0, 0).
Therefore,  $(\vec x \times \vec y)\cdot\vec z \ne |\cdots|$ in Matlab.
Mathematica conjugates neither. Maple conjugates the dot product (though you have the option to nonconjugate it) but not the cross product.
https://www.maplesoft.com/support/help/Maple/view.aspx?path=LinearAlgebra/DotProduct
https://www.maplesoft.com/support/help/Maple/view.aspx?path=LinearAlgebra/CrossProduct
My guess is that Scilab does not conjugate. Somebody hopefully certifies/corrects this paragraph.
This answer builds on the shoulders of the answer by Marc van Leeuwen on this same page.
