What is really the purpose of $i$? We started talking about imaginary numbers again this year and asked this question in class, but nobody could really give a straight answer. So if anyone could tell me the real reason we have imaginary numbers that would be great! :)
 A: The real numbers you are familiar with (positive, negative, and zero) are incomplete.  They are part of a larger system, the complex numbers, and many matters that are mathematically strange or puzzling become clear when you view them with the full context.  Many difficult problems become easy and many impossible problems become solvable.
Here is a short example.
For many reasons we spend a lot of time studying polynomial equations.  For example, $x^2+x-6=0$ (this is an equation of the “2nd degree” because the highest power of $x$ is $x^2$) or $23x^5-12x^3+x+127 = 0$ (this is an equation of the “5th degree”).  For equations of second degree and higher, there can be more than one solution.  For example,  $x^2+x-6=0$  is solved by both $x=2$ and by $x=-3$.
Sometimes, though, these equations appear to have no solution.  For example, $x^2+4x+5=0$ has no solutions.  One can ask how to tell when an equation has solutions, and how many it has.  It's not hard, for example, to show that any equation of odd degree must have at least one solution.  But it is hard to make much progress past that point.
If you look at the whole context, which is the complex numbers, the answer is rather simple.  An equation of the $n$th degree always has a solution, and in fact it always has exactly $n$ solutions! For example, the equation $x^2+4x+5=0$ I mentioned earlier as having “no solutions” is solved by $-2+i$ and by $-2-i$.
The polynomials don't care that you think the complex numbers are strange.  They are doing their own thing, and to them, the complex numbers are what is important and the real numbers are just a weird little province.  If we want to deal with polynomials, we have to visit them in the place where they live.  This is the complex numbers.
Here is a longer example.
There is a certain infinite series, $$1+x^2+x^4+x^6+\ldots$$  which (for reasons I don't expect you to appreciate right now) looks as though it should add up to $$\frac1{1-x^2}.$$  That is, you pick $x$, and then as you add up more and more terms of the series, the sum gets closer and closer to $\frac1{1-x^2}$.  For instance, $$1 + \left(\frac13\right)^2 + \left(\frac13\right)^4 + \left(\frac13\right)^6 = 1.12483…$$ which is already quite close to $\frac1{1-\left(\frac13\right)^2} = \frac98 = 1.125$, and the following terms of the series make up the missing difference.  You should try this yourself with $\frac12$.
So the series does add up the way we expect—if the absolute value of $x$ is less than $1$.    But outside that range it does not work. (Try it for some $x$ bigger than $1$; it will be obvious that it is completely wrong.) Why not?
This is not just an idle question.  This sort of infinite series is an important part of a method for solving differential equations, which are crucial to many problems of physics, architecture, and engineering.  We need to understand when this method works and when it doesn't, so that our buildings don't fall down.
The reason that $\frac1{1-x^2}$ fails  when $x$ is too large is subtle, but the basic cause is that $\frac1{1-x^2}$ is undefined at $-1$ and at $1$, because you would have to divide by zero.  As you move past those undefined points something breaks.
Using the same methods, we can find an infinite series $$1-x^2+x^4-x^6+\ldots$$ that adds up to $$\frac1{1+x^2}.$$  This also works—but again, only if $x$ is between $-1$ and $1$.  Why does it fail when $x$ is outside that range?  The explanation of the previous paragraph doesn't make sense any more, because unlike $\frac1{1-x^2}$, the function $\frac1{1+x^2}$ is not undefined at $-1$ or at $1$.  It is defined everywhere, so why doesn't the series work everywhere?
To understand what is really going on, you have to look at the complex numbers.  If we think of the complex numbers as being arranged in a plane, it turns out that the series for $\frac1{1-x^2}$ works for any complex number inside a certain circle.  The circle contains not just the real numbers whose absolute value is less than 1, but all the complex numbers whose absolute value is less than 1. The circle passes through  $+1$ and $-1$, and the undefinedness of $\frac1{1-x^2}$ at those two points spoils the series everywhere outside this circle.
And now the failure for the series for $\frac1{1+x^2}$ makes sense: the series works anywhere inside that same circle.  Why does it break on that circle when $\frac1{1+x^2}$ is perfectly well-defined at $1$ and at $-1$?  Because the circle also goes through $i$ and $-i$, and at those points $\frac1{1+x^2}$ is not well-defined.  It breaks in exactly the same way, for exactly the same reason!
The two functions that looked so different are actually almost exactly the same once you look at the entire picture.  The complex numbers are the entire picture.
A: The real reason? That sounds like you want a one-liner that you might not have heard before.
Take out the graph paper.
What happens if you keep applying $i$ to $1$? i.e, $\;i \times 1$, $\;i \times (i \times 1)$, $\;i \times (i \times (i \times 1))$, etc.
Apply $i$ to $1$ and you get $i$.
Apply $i$ to $i$ and you get $-1$.
Apply $i$ to $-1$ and you get $-i$.
Apply $i$ to $-i$ and you get $+1$.
So repeated application and you get to watch $1 \mapsto i \mapsto -1 \mapsto -i \mapsto 1 \dots$ 
Exercise: What happens if you keep applying $i$ to $1 + i$.

The real reason for $i$ is you get to rotate stuff by 90° using number
  multiplication.

There is nothing imaginary about rotations!
A: The real purpose is be able to carry out mathematical operations in an unrestricted manner. 
Why have negative numbers ?
So you can subtract: $2-3=-1$
Why have fractional  numbers ?
So you can divide: $2\div 3=\frac{2}{2}$
Why have imaginary numbers ?
So you can take square roots: $\sqrt{-1}=i$.
It turns out to be a minor miracle that with imaginary numbers ALL algebraic equations, eg, $x^7+4x^6+3x^5+76x^4+14x^3+12x^2+89x+100=0$ have (possibly imaginary) solutions.
The the range of applicability of the operations is greatly extended.
A: Historically, the imaginary and complex numbers were introduced for the resolution of the cubic equation: in the case of three real roots, you can't really do without an excursion in the complex numbers, even though the final results are real !
But a deeper reason is the Fundamental Theorem of Algebra, stating that every polynomial has at least one root: for it to be true, you must admit complex roots as well. Strange as it may sound, the simplicity of the theorem is so powerful that mathematicians prefer to accept complex numbers rather than dropping the fundamental theorem. This makes the set of complex numbers $\mathbb C$ a more natural domain than $\mathbb R$ for algebraic operations.
As you will later see, complex numbers allow numerous shortcuts in some computations and the theory of complex functions turns out to be rich and useful.

For the sake of illustration, the Euler's formula unifies the exponential and the trigonometric functions:
$$e^{ix}=\cos x+i\sin x.$$
As an application, consider
$$(e^{ix})^3=(\cos x+i\sin x)^3=\cos^3x-3\cos x\sin^2x+i(3\cos^2x\sin x-\sin^3x)\\
=e^{i3x}=\cos 3x+i\sin 3x.$$
This establishes in a single go the two formulas
$$\cos 3x=\cos^3x-3\cos x\sin^2x,\\\sin3x=3\cos^2x\sin x-\sin^3x$$
which are more painful to get at by other means.
A: @Yves Daoust gave great answer in the Algebric field and showed how powerful it can be so I won't add more in that regard.
Imaginary, or complex, numbers in physics.
In quantum physics there are systems of superposition, let's say you have 2 states, $A,B$ and let's say $a$ is the probability for $A$ to be the result $b$ is the probability of $B$ to be the result.  To be more clear and follow how it is written in reality we will say $|A\rangle,|B\rangle$ instead of $A,B$.
Now the final position will be $|\psi\rangle$. One will think that $|\psi\rangle=a|A\rangle+b|B\rangle$ but in reality the probability is $a^2,b^2$, so we can also say it as say $|\psi\rangle=a|A\rangle-ib|B\rangle$(because $b^2=-(ib)^2$)
This is SO useful, think about it, this little, that look almost meaning, showing as connection between quantum physics and waves!
I won't get more into this and why it is so useful because it requires a little background in physics.
There are  a lot of complex numbers in quantum physics, apparently quantum physics has almost nothing without complex numbers, for example 
Schrödinger equation: $i\hbar\dfrac{\partial}{\partial t}\Psi(\mathbf{r},t)=\hat H \Psi(\mathbf{r},t)$ where $\hbar$ is the reduced planck constant.
I can keep going, read about quantum physics if this is interest you
A: Many here pointed out the historical reasons for the use of complex numbers and in particular the fundamental theorem of algebra that says that every polynomial has a root over the complex numbers.
But except for the answer of @MikeMathMan, the geometric aspect was not elucidated and so I would like to do that in more detail.
First of all, to say that $i$ is just another name for $\sqrt{-1}$ where $-1\in\mathbb{R}$ is in my opinion at least a bit confusing - instead, $i$ is something that can not be entirely comprehended in terms of a square root of a real number if one starts out with the usual understanding of the square root as a map $\sqrt{~}:\mathbb{R}\rightarrow\mathbb{R}$ because a whole new dimension is necessary for it to make sense. So it would be better to say that $\sqrt{-1}$ acts as a label of what $i$ means. In the following, I will explain more clearly what this means:
It is useful to start by noting that $\mathbb{C}$ is not just isomorphic to $\mathbb{R}^2$ although there is an isomorphism between the underlying vector spaces. $\mathbb{C}$ carries more structure  because it is an algebra, that means, a vector space equipped with a product that sends two complex numbers to a complex number. 
\begin{equation}
(x_1+iy_1)(x_2+iy_2)=(x_1x_2-y_1y_2)+i(x_1y_2+x_2y_1)
\end{equation}
So $\mathbb{C}$ is $\mathbb{R}^2$ with this product. 
However, every associative
  algebra can be represented
  on a subspace of AutoMorphisms
  over some vector space
  (because these also form
  an associative algebra). Thus,

There is an IsoMorphism from the complex numbers to a subspace of the real 2x2 matrices preserving complex multiplication.

Explicitly, by comprehending the two matrices
  \begin{equation}
    \mathbf{1} =
    \begin{pmatrix}
      1&0\\
      0&1
    \end{pmatrix},\qquad
    \mathbf{i} =
    \begin{pmatrix}
      0&-1\\
      1&0
    \end{pmatrix}
    \label{eq:complexNumberMatrices}
  \end{equation}
  as the basis of a two dimensional subspace $\Lambda$ of the 4-dimensional space of 2x2 matrices,
  we observe that $\mathbf{i}^2 = -\mathbf{1}$, just as $i^2=-1$, which suggests the
  natural isomorphism $\varphi$ that maps $\mathbb{C}$ to $\Lambda$ by
  \begin{equation}
    \varphi(x+iy):=x\mathbf{1}+y\mathbf{i}, \qquad x,y\in \mathbb{R}.
  \end{equation}
  In particular, addition and multiplication defined on $\mathbb{C}$ are carried over to $\Lambda$:
  \begin{equation}
    (x_1\mathbf{1}+y_1\mathbf{i})(x_2\mathbf{1}+y_2\mathbf{i}) = (x_1x_2-y_1y_2)\mathbf{1}+(x_1y_2+x_2y_1)\mathbf{i}.
  \end{equation}
  Division corresponds to the multiplication of the inverse of the
  matrix expressions. This illustrates that the complex numbers can actually be understood as linear operators instead of
  normal numbers. In contrast, $\mathbb{Q}$ or
  $\mathbb{R}$ can be viewed as spaces of linear maps that
  act on $themselves$ by multiplication.
In my opinion, this makes much clearer 
   that $\mathbf{i}$ can be viewed as the operator that turns 2-dimensional vectors around 90 degrees. (As addressed by @MikeMathMan)
And, to close the explanation of why it is confusing to view $i$ as just a label of $\sqrt{-1}$, let us look at the matrix equation: $\mathbf{x}\cdot\mathbf{x}=-\mathbf{1}$. There is a solution, namely in form of $\mathbf{i}$ - and thus we can $write$ this solution as $\sqrt{-\mathbf{1}}$ but observe carefully how the square root gets a new significance here: it is a square root of an operator, not just a real number. So, if one defines $i$ and the complex plane, then $\sqrt{-1}$ is much more a label for what $i$ means than the other way round. If $-1$ is a real number, then $\sqrt{-1}$ can not be comprehended in terms of anything on the real number line - that is why another dimension is necessary. 
In particular, one can view the fundamental theorem of algebra as a theorem over the polynomials over the subspace $\Lambda$ of 2x2-matrices - on this matrix space, every polynomial has a solution.
Using this inside, we can connect our new view with the idea of Sophus Lie who understood that to study rotations, it suffices
to study rotation through infinitesimal angles and this will also make clearer why relations like $e^{i\theta}=\cos(\theta)+i\sin(\theta),~
  \theta\in \mathbb{R}$ hold: 
Rotations in n-dimensional space are normally defined by
  the matrix relation $\mathbf{R}\mathbf{R}^T:=\mathbf{1}_n$,
  where $\mathbf{R}$ is an $n\times n$-matrix. Evolving this $\mathbf{R}$ around a small value (i.e. finding only
  the matrix for
  infinitesimal rotations around the angle $\theta/N$, $N\rightarrow\infty$)
  results in the equation $\mathbf{1}_n=(\mathbf{1}_n+(\theta/N) \mathbf{J})(\mathbf{1}_n+(\theta/N) \mathbf{J}^T)=
  \mathbf{1}_n+(\theta/N)(\mathbf{J}+\mathbf{J}^T)+\mathcal{O}(\theta/N)^2$
  which is fulfilled if $\mathbf{J}=-\mathbf{J}^T$. The solution of that equation in 2 dimensions is $\mathbf{J}=\pm\mathbf{i}$,
  that means that we discovered that the $i$ of $\mathbb{C}$ actually is not only a rotation around 90 degrees
  but when multiplied with an infinitesimal parameter $\theta/N$ it becomes the infinitesimal rotation operator.
This can be understood geometrically in terms of the picture below.
  As Lie pointed out, when such an infinitesimal operation is applied infinitely many times,
  the result is the full rotation because $\mathbf{R}=\lim_{N\rightarrow\infty}\big(\mathbf{1}+\frac{\theta \mathbf{i}}{N}\big)^N
  =\exp(\mathbf{i}\theta)$.
  Thus when defining the exponential as a power series of matrices, we obtain the rotation matrix.

Of course all these arguments can be formulated with $i$ as well instead of $\mathbf{i}$ but then the geometric significance of $i$ becomes less clear.
A: They are useful, not just for solving quadratic equations, but for many algebraic equations.  Legend has it that the mathematicians of the Italian Renaissance (in particular, Cardano, Tartaglia, and Viete), noticed that if you allowed yourself an imaginary number $i$ with $i^2=-1$, then you could construct formulas for finding roots to cubic equations.  They were ambivalent about the use of such numbers (hence the names real and imaginary), but if the roots are real, these imaginary methods will still find them.  I think this is how they caught on.
For a more scholarly telling of the story, you might look at An Imaginary Tale: The Story of $\sqrt{-1}$ by Paul J. Nahin.
A: The complex numbers as a set are just pairs of real numbers, like $(1,4)$, which we usualy write as $1+4i$ and you will see why: 
We can define addition between them as you would expect, if $z_1 = (x_1,y_1) = x_1 +y_1 i$ and $z_2 = (x_2, y_2)= x_2 +y_2 i $ then $$z_1+ z_2 = (x_1 +x_2, y_1+y_2)= (x_1 +x_2) + (y_1+y_2)i$$
Nothing "unsual" so far. The difference comes when we define multiplication between them as $$ (x_1 , y_1) \cdot (x_2,y_2) = (x_1 x_2-y_1 y_2, x_1 y_2 + y_1 x_2)$$
See what happens then, when you multiply $(0,1) \cdot (0,1)=(-1,0)$
So we can DEFINE $i=(0,1)$ and thus we can write numbers like $(1,4)$ as follows :$$(1,4)= 1*(1,0) + 4*(0,1)=1 +4i$$  Then what we saw is that $ i \cdot i = i^2 =-1 +0i=-1$.
There is nothing imaginary about them, we just define multiplication in a certain way and it turns out it's pretty convenient.
