What's the use of quadratic forms? Starting with the abstract concept of a vector space, I can see why we'd want to add some structure to be able to perform useful operations.  For instance if we add a metric/ norm to a vector space we can talk about distances.  If we add an inner product to a vector space we can talk about angles.  These two operations also give us a bunch of inequalities (like Cauchy-Schwarz) that we get as well.
But I don't see the point of equipping our vector space with some degree two polynomial.  What does that get us?  Is there some geometric meaning to it (like how we got distance and angle from norm and inner product)?
 A: Just want to mention an application. Quadratic forms appear in optimization problems of multi-variable functions. Also, the idea of positive definiteness and negative definiteness are based on the quadratic form concept. This gives rise to a test in order to distinguish the extreme points, namely Maximum, Minimum, and Saddle points. In fact, quadratic forms are the building blocks of the multi-variable optimization.
Just to write some formulas, consider the scalar field $f:\mathbb{R}^n \to \mathbb{R}$ denoted by $f({\bf{x}})$. Then writting the Taylor expansion at ${\bf{x}}={\bf{x}}_0$ will lead to
$$f({\bf{x}})=f({\bf{x}}_0)+\nabla f({\bf{x}}_0) \cdot ({\bf{x}}-{\bf{x}}_0) + \frac{1}{2} ({\bf{x}}-{\bf{x}}_0) \cdot \nabla \nabla f ({\bf{x}}_0) \cdot ({\bf{x}}-{\bf{x}}_0) + \cdot \cdot \cdot$$
Next, for ${\bf{x}}_0$ to be an extreme point one requires that
$$\nabla f ({\bf{x}}_0) =0$$
and to study the type of extreme point one should study carefully the quadratic form 
$$Q({\bf{x}})=\frac{1}{2} ({\bf{x}}-{\bf{x}}_0) \cdot \nabla \nabla f ({\bf{x}}_0) \cdot ({\bf{x}}-{\bf{x}}_0)$$
A: Another place where bilinear and quadratic forms appear naturally and have geometric meaning is in algebraic and differential topology. If $M$ is a compact connected oriented $2n$ dimensional manifold, the wedge product induces a bilinear form on the vector space $H_{\mathrm{dR}}^n(M)$ of $n$-dimensional de Rham cohomology classes on $M$. Using Poincaré duality, one can interpret this bilinear form as computing generic signed intersections between $n$ dimensional submanifolds of $M$.
I can demonstrate this intuitively for the case $M$ is a two-dimensional torus. Consider the following image (taken from  the Wolfram Mathworld page on homology intersection):

For the torus, $H_{\mathrm{dr}}^1(M)$ is a two dimensional real vector space and we can choose a basis $\mathcal{B} = (v_1, v_2)$ for $H_{\mathrm{dr}}^1(M)$ under which $v_1$ corresponds to the blue circle in the picture and $v_2$ corresponds to the red circle. On $H_{\mathrm{dr}}^1(M)$ we have a bilinear form $g$ which encodes the intersections between the circles. We have $ g(v_1, v_1) = 0 $ which corresponds to the fact that if we perturb the blue circle a little, the intersection between the original blue circle and the perturbed circle is zero. Similarly, we have $g(v_2, v_2) = 0$. However, we (can choose the basis so that) have $g(v_1, v_2) = 1$ which corresponds to the fact that the blue circle and the red circle intersect at a single point (counted with a plus sign) and even if we perturb them a little, they will still generically intersect at a single point. If we change the order of $v_1$ and $v_2$, this doesn't effect the geometric intersection but does change the sign and so $g(v_2, v_1) = -1$. Thus, the intersection form $g$ is a bilinear form on $H_{\mathrm{dr}}^1(M)$ represented by the matrix
$$ [g]_{\mathcal{B}} = \begin{pmatrix} 0 & 1 \\ -1 & 0 \end{pmatrix}. $$
In this case, $g$ is an anti-symmetric bilinear form but if the dimension of $M$ is divisible by $4$, it will be a symmetric bilinear form (corresponding to a quadratic form). The form $g$ encodes information about submanifolds sitting inside $M$ and is an important invariant of $M$. It can be used to show that two manifolds are not homeomorphic but showing that corresponding bilinear forms are not equivalent. You can read a bit more about this here.
A: Outside of characteristic $2$ fields, there is a correspondence between quadratic forms and symmetric bilinear forms. (You can still say something in the characteristic $2$ case, you just have to be more careful.)
So in some sense, what you must have already learned with the run-of-the-mill dot product of vectors actually generalizes in a very nice way to include more interesting spaces. 
In the case of $\Bbb R$, say, this allows vectors with zero length and negative length. In the case of $\Bbb R^3$ with $Q((x,y,z)=x^2+y^2-z^2$, you get a indefinite but nondegenerate bilinear form on the space. By specifying a hyperboloid sheet in this space, you can model the hyperbolic plane.
I think if you are interested in this, I would like to recommend Kaplanksy's Linear algebra and geometry to learn about quadratic and bilinear forms, and then find a basic exposition on differential geometry that explains the roots of the extrema tests (mentioned by H.R.) in terms of quadratic forms.
In differential geometry, the first and second fundamental forms incorporate bilinear/quadratic forms. Not just a single form, mind you, but these actually speak of a family of forms, one for each point of some surface in space. The forms can differ from point to point depending on the nature of the space, but at any given point, the form gives information about the shape of the surface near the point.
