The commutator product as a replacement for the cross and wedge product in geometric algebra? From an axiomatic approach to geometric algebra, the wedge product of two vectors $a$ and $b$ is typcially defined as the antisymmetric $a \wedge b = \frac{1}{2}(ab - ba)$, where $ab$ is the geometric product of $a$ and $b$, and is used as the "replacement" for the cross product in 3 dimensions. However, the commutator product for two arbitrary multivectors $A$ and $B$ is also defined to be $A \times B = \frac{1}{2}(AB - BA)$, where $AB$ is the geometric product of $A$ and $B$. For two vectors this is the same as the wedge product - a bivector, a pseudovector in 3 dimensions. For vector $a$ and bivector $B$, the commutator product $a \times B$ yields another vector $d$, and substituting $B = b \times c$ for two vectors $b$ and $c$, we get $d = a \times (b \times c)$, which is the triple vector product, but using the commutator product instead of the cross product. 
The dot product is likewise defined from the axiomatic approach as the symmetric $a \cdot b = \frac{1}{2}(ab + ba)$ which could also potentially be extended in the same manner as the commutator to general multivectors $A$ and $B$. In this manner the symmetric dot product of vector $a$ and a bivector $B$ is a trivector $\Gamma$, which in three dimensions is simply a pseudoscalar $\gamma I$. Substituting $B = b \times c$ into this equation, we get another equation $\gamma I = a \cdot (b \times c)$, which is very similar to the triple scalar product using the traditional vector dot and cross products, except yielding a pseudoscalar instead of a scalar. 
So with all these similarities between the commutator and the vector cross product, is it possible to use the commutator product used as the replacement for the cross product in geometric algebra in place of the wedge product? And likewise for the symmetric product extension of the dot product $A \cdot B = \frac{1}{2}(AB + BA)$ in place of the typical contraction extension? What are the potential disadvantages of using the commutator and the symmetric dot product in place of the wedge product and contraction dot product in geometric algebra, especially in 3 dimensions?
 A: Welcome to MSE! 

So with all these similarities between the commutator and the vector cross product, is it possible to use the commutator product used as the replacement for the cross product in geometric algebra in place of the wedge product? 

First, the vector cross product and the geometric algebra commutator use the same symbol $\times$ for their operation. This similarity, not mentioned in your post, could be potentially very confusing when most of the mathematicians and physicists learn vector algebra before they learn geometric algebra, as the commutator of two vectors (as you said) would yield a bivector, while the cross product would only yield a vector. 
Another subtle difference between the vector cross product and the commutator that could cause confusion is that the result from the triple vector cross product is the negative of the corresponding triple product for the commutator in $G^{3,0}$. For vectors $a$, $b$, $c$, $d$, and $e$, let $d = a \times (b \times c)$ and $e = b \times c$, where $\times$ represents the vector cross product. Now if $\times$ were to be the geometric algebra commutator, $b \times c$ would instead equal $Ie$, and $a \times (b \times c) = a \times Ie = I(a \times e) = I^2 d = -d$, due to the properties of the pseudoscaler in $G^{3,0}$, causing another potential source of confusion and error.
Due to the possible confusion between the two operators, in geometric algebra the commutator is typically only used for bivectors, while the outer product/wedge product is used for vectors. Perhaps in a future when vector algebra has been superseded by geometric algebra, the commutator can be more widely used (but see below).

And likewise for the symmetric product extension of the dot product $A \cdot B = \frac{1}{2}(AB + BA)$ in place of the typical contraction extension? 

In a geometric algebra of dimension less than three, this definition would yield a pure-grade multivector, but once you reach four dimensions, this product of two bivectors would yield a mixed-grade multivector, one of grade 0 and another of grade 4. Typically, we only wish to have one of the two multivector terms - the inner product/contraction would yield the grade 0 term, while the outer product would yield the grade 4 term. Thus, for GA applications such as 3D Conformal Geometry ($G^{4,1}$) or Spacetime Algebra ($G^{3, 1}$), the contraction and outer product would be more useful in that sense. 

What are the potential disadvantages of using the commutator and the symmetric dot product in place of the wedge product and contraction dot product in geometric algebra

There are actually two contractions, the left contraction and the right contraction, which do have the less-commonly-used symbols $\lfloor$ and $\rfloor$ as well, but due to historical reasons and the relationships with the vector dot product and the inner product in linear algebra, the $\cdot$ symbol is typically used for the left contraction. So, technically, you're after defining two new binary operations in geometric algebra, which could be used alongside the contractions and the outer product. 
But as a replacement for the contractions and the outer product? One of the big advantages of using geometric algebra is that there is a clear intuition between the algebraic objects and operators in geometric algebra and the corresponding geometric objects in the defined space of any dimension. The contractions and outer product have a distinctive geometric interpretation (as do blades) - the contraction of two blades $A$ and $B$ being the part of $A$ most unlike $B$, according to Leo Dorst in his textbook Geometric Algebra for Computer Science, while the outer product of $A$ and $B$ is the blade $C$ representing the subspace whose basis is the union of the the bases of the subspaces representing $A$ and $B$. But it is unclear what the geometric interpretation of either the commutator or your defined symmetric dot product should result in. Thus using your two products in place of the ones currently used in geometric algebra would lose some of the intuitive geometric interpretation of geometric algebra that makes it such a powerful tool in geometry, physics, and engineering. 
The commutator product is important for bivectors in geometric algebra primarily not for its associations with geometry, but rather because the bivectors equipped with the commutator product are associated with particular Lie Algebras from abstract algebra, which are used greatly in more advanced physics. 
So, while it is possible to use the commutator in place of the wedge product, the wedge product offers many more advantages to geometric algebra compared to the commutator. 

especially in 3 dimensions?

I make a separate case for three dimensions, as it seems that, if a different symbol were to be used for the commutator, your algebra does seem to be a midway point between traditional vector algebra and full-blown $G^{3,0}$. The main issue with the output of the cross product is that it fails to distinguish between polar and axial vectors, or vectors and pseudovectors, but by introducing an commutative pseudoscalar (complex imaginary) unit whose square is negative one, and defining pseudovectors as the product of a vector and the pseudoscalar, that issue with the output of the cross product is resolved, and furthermore it highlights a link between quaternions and vector algebra. However, this is only the case in three dimensions, and does not generalise to other dimensions, as geometric algebra is intended to, and so suffers from the same limitations as traditional vector algebra. 
Edit: That was quite the long response. I wasn't expecting that.
