When are complex numbers insufficient? Rationals can't solve $x^2=2$, and reals can't solve $x^2=-1$. Is there any problem that cannot be solved by complex numbers but can be solved by non-standard numbers?
Every polynomial with coefficients in $C$ can be solved by numbers in $C$, can every equation* be solved by numbers in $C$?
*(that can not be simplified to $1=0$)
 A: As $\mathbb{C}$ is commutative, non-commutative problems cannot be solved in it.
E.g. $A \cdot B - B \cdot A = I.$
Equations like this are central to Quantum Mechanics and Lie Algebras.
However, matrices usually are non-commutative, so matrices over $\mathbb{C}$ or $\mathbb{R}$ are able to solve those equations. As can quaternions among others.
EDIT: You say that you are looking for non-standard numbers. May I suggest to have a look at Quaternions? They are the logical next step after $\mathbb{C}$. The Wikipedia entry might be a good starting point. Quaternions are a bit out of fashion, but theoretically and historically they are important.
A: The problem, "find a number strictly greater than zero but strictly less than every number $1/n$ with $n=1,2,3,\dots$" cannot be solved in the complex numbers, but can be solved in the non-standard numbers. 
A: I would like to respond to the question "Is there any problem that cannot be solved by complex numbers but can be solved by non-standard numbers?"  Since one of the tags is "nonstandard analysis", I will interpret this as applying the the nonstandard numbers in that theory, namely the hyperreals.  Back to the question: one important problem that the hyperreals allow one to solve, is relating the derivative of a function to ratios of infinitesimals, as it was done by Leibniz, a co-founder of infinitesimal calculus.
