Why is better to work with the spectrum of prime ideals than with the maximal one, for example in the definition of affine scheme. When we have an algebraic variety we can identify the points of the variety with maximal ideals of the coordinate ring. 
I would like to know why it is more natural to define the main structure of the theory of schemes, the affine scheme, with prime ideals and not with maximal ones. 
When Grothendieck was creating theory of schemes, why did he decide to work with the normal spectrum instead of the maximal one?
(As you can see I dont have an strong background of Algebraic Geometry, I just want to have some intuition) 
In which sense the schemes generalize the notion of variety and why is better to work with this notion? 
 A: This is the moment in which I should give a very well documented answer, full of critical examples, very long and well explained. I'll try, but it's been a while I am not in algebraic geometry, so I ask for tolerance :)
If you think of maximal ideals as points in your variety, prime ideals represents irreducible subvarieties: lines, curves, planes...  So when we say that a maximal ideal can be mapped to prime ideal which is non maximal, we are saying that a rational function can, in principle, send a point to a line. What the hell does this mean? Let's see an example.
Take a ring $A$ and a prime $p$ which is not maximal: you can imagine $p$ represent a line. Consider the localization $A_p$. Now $pA_p$ becomes maximal in $A_p$, and the inclusion $i:A\to A_p$ sends via preimage $pA_p$ to the non maximal ideal $p$. See? We sent a "point" to a line, but we all know this is a lie. Let me explain better.
When you take a local ring $R$ there is only one maximal ideal, but we don't want to think this to be a one-point variety: as you have seen above, the richness of this variety is in the "neighborhood" of the maximal ideal.
Also, since we cannot really talk about disks and balls in the euclidean space as we do in classical geometry (this would require using inequalities, which are not defined for arbitrary fields), we need a notion of being close for defining a topology. This is achieved by declaring subvarieties to be the closed sunsets. That's not finer as the classical topology, but we can live with that. For example, here it is a thing that I struggled with: take a plane and remove three general lines; is this connected? In classical topology of course it is not, but in algebraic geometry there is no way you can disconnect it, no matter which (robust) notion of connection you take. This emanates from the fact that you have to use lines and not segments to connect stuff; also, lines can be punctured.
Let me also state a final remark which is crucial as a springboard to schemes. Local to global principles play a crucial role thorough all geometry, and in a special way in algebraic geometry.
Schemes are objects which locally looks like affine algebraic varieties, and this often permits to do local-to-global arguments (i.e. reduce to the case in which you have $Spec A$ and then use commutative algebra). In the same way, noetherian rings locally looks like local rings; this means that a property for a ring $A$ is equivalent to the same property for the ring $A_p$, where $p$ ranges over all primes. Not all criteria holds if you substitutes all primes with maximal.
Of course, not all properties of schemes are local (see connectedness) and not all properties of rings are local, but nevertheless this is probably something that was used a lot and well established at the time schemes were introduced. It would be nice if someone with historical knowledge could comment, though.
A: In this thread
https://mathoverflow.net/questions/377922/building-algebraic-geometry-without-prime-ideals/378961#378961
you will find a construction of a locally ringed space $(X^m, \mathcal{O}_{X^m})$
from any scheme $X$ of finite type over a field $k$ or the ring if integers $\mathbb{Z}$. The topological space $X^m$ is by definition the space of closed points in $X$ with the induced topology. This construction allows you to speak of nilpotent elements without using prime ideals.
Here I include the post linked to above:
In this thread
https://mathoverflow.net/questions/55244/why-must-nilpotent-elements-be-allowed-in-modern-algebraic-geometry/378811#378811
you find the following construction: Let $k$ be a field or the ring $\mathbb{Z}$ of integers and let $X$ be a scheme of finite type over $k$. By this we mean $X$ has a finite open affine cover $X=\cup_{i=1}^n Spec(A_i)$ where $A_i$ is a finitely generated $k$-algebra for all $i$.
Definition 1. Let $X^m$ be the set of closed points in $X$ with $u:X^m \rightarrow X$ the canonical inclusion map and with the induced topology. Let $\mathcal{O}_{X^m}:=u^{-1}(\mathcal{O}_{X})$ be the topological inverse of $\mathcal{O}_X$.
It follows the pair $(X^m, \mathcal{O}_{X^m})$ is a locally ringed space. In the above thread I explain why $X^m$ is a locally ringed space that is "similar" to a "classical algebraic variety" as defined in Hartshornes book in chapter I as the set of zeros of an ideal in a polynomial ring in a finite set of variables. The difference is that Hartshorne starts with a fixed algebraically closed field $k$ and an ideal $I:=\{f_1,..,f_n\} \subseteq B:=k[x_1,..,x_n]$ in a polynomial ring $B$ in a finite set of variables.
Example: Hartshorne defines $V(I)$ as the "set of $n$-tuples" $\{t:=(t_1,..,t_n)\in k^n\}$ such that $f(t_1,..,t_n)=0$ for all poynomials in $I$. Hence Hartshornes $n$-tuples $t$ have their coefficients in the base field $k$. Since the field $k$ is algebraically closed it follows the set of maximal ideals $\mathfrak{m}$ in $X^m$ all have residue field $k$. Hence for an algebraically closed field $k$ it follows $X^m=V(I)$ if $X:=Spec(A)$ and $A:=B/I$. The above Definition 1 makes sense for any Hilbert-Jacobson ring $k$. You need the property that any prime ideal $\mathfrak{p}$ in $k$ is the intersection of maximal ideals.
Example. Let $A:=\mathbb{R}[x]$ and let $C:=Spec(A)$. It follows $C^m$ is the set of irreducible polynomials in $A$. A polynomial $p(x)$ in $A$ is irreducible iff $p(x)=x-r$ with $r\in \mathbb{R}$ or $p(x)=(x-z)(x-\overline{z})=x^2-4ax+a^2+b^2$ with $a,b\in \mathbb{R}$ and $b \ne 0$. In the above thread I explain how you may use nilpotent elements to Taylor expand sections of $\mathcal{O}_{C^m}(U)$. To explain Taylor expansion for sections of sheaves to students using real curves is "easier to understand" and "more intuitive". Sometimes students have problems understanding the field of complex numbers.
The structure sheaf $\mathcal{O}_{X^m}$ has the property
P1. $\Gamma(X^m, \mathcal{O}_{X^m})=i^{-1}(\mathcal{O}_X)(X^m):=lim_{X^m \subseteq U}\mathcal{O}_X(U) \cong \Gamma(X, \mathcal{O}_X)$,
hence $\mathcal{O}_{X^m}$ and $\mathcal{O}_X$ have the same global sections. Hence if $X:=Spec(A)$ it follows $\Gamma(X^m, \mathcal{O}_{X^m})=A$ and you recover the ring $A$ from the locally ringed space  $(X^m, \mathcal{O}_{X^m})$.
Example: If $S:=A[x_1,..,x_n]/I$ where $I$ is a homogeneous ideal and $A$ is a Hilbert-Jacobson ring we may define $X^m \subseteq X:=Proj(S)$. If $\mathcal{E}$ is any finite rank locally trivial  $\mathcal{O}_X$-module, we may define $\mathbb{P}(\mathcal{E}^*)^m \subseteq \mathbb{P}(\mathcal{E}^*)$.
If $I \subseteq A\otimes A$ is the ideal of the diagonal and if
$\mathcal{P}(l):=Spec(A\otimes A/I^{l+1})$
we may define $\mathcal{P}(l)^m$. It has the property that
$\Gamma(\mathcal{P}(l), \mathcal{O}_{\mathcal{P}(l)})=A\otimes A/I^{l+1}$.
The canonical surjective map $m: A\otimes A/I^{l+1} \rightarrow A$ gives a one-to-one correspondence between $\mathcal{P}(l)^m$ and $Spec(A)^m$. Hence $\mathcal{P}(l)^m$ has the same points as $Spec(A)^m$ but it has non-trivial nilpotent elements in the structure sheaf - it is a "classical algebraic variety" with a non-reduced structure sheaf.
Comment: "Of course, there are some problems with this approach: The class of all fields is not a set. Technically, we can limit ourselves to some very large set of "test fields". So this can be swept under the rug."
Remark: The ring $A$ can be an arbitrary finitely generated $k$-algebra and it can be non-reduced. Hence this gives a way of introducing nilpotent elements for "classical algebraic varietiwith this. Moreover you don't use prime ideals, hence the construction gives an answer to the original question: Yes you may do this if the base ring is  a finitely generated ring over a field or the integers $\mathbb{Z}$.
Example. Let $(A,\mathfrak{m})$ be a local ring and let $X:=Spec(A)$ with $X^m:=\{\mathfrak{m}\}$ the one-point space with inclusion map
$i: X^m \rightarrow X$
and structure sheaf $\mathcal{O}_{X^m}:=i^{-1}(\mathcal{O}_X)$. By definition
$\Gamma(X^m, \mathcal{O}_{X^m}):=lim_{\mathfrak{m}\in U}\mathcal{O}_X(U)\cong \mathcal{O}_{X,\mathfrak{m}} \cong A_{\mathfrak{m}}\cong A$
since $A-\mathfrak{m}$ consists of units. Hence the global sections of the structure sheaf of $X^m$ recover the ring $A$. It follows you recover the ring $A$ from the locally ringed space $(X^m, \mathcal{O}_{X^m})$, and this construction can be done for an arbitrary local ring.
Let $\phi: (A, \mathfrak{m})\rightarrow (B, \mathfrak{n})$ be a map of local rings ($\phi^{-1}(\mathfrak{n})=\mathfrak{m}$). It follows we get a map of ringed spaces
$(\phi^m, (\phi^m)^{\#}): (X^m, \mathcal{O}_{X^m})\rightarrow (Y^m, \mathcal{O}_{Y^m})$.
The map
$(\phi^m)^{\#}: \mathcal{O}_{Y^m}\rightarrow (\phi^m)_*\mathcal{O}_{X^m}$
recovers the map $\phi$. Here $X^m:=Spec(B)^m$ and $Y^m:=Spec(A)^m$.
Hence the ringed space $(X^m, \mathcal{O}_{X^m})$ recovers the local ring $(B, \mathfrak{n})$ and maps of local rings. As a topological space $X^m$ is not very interesting, but the
algebraic structure is captured by the structure sheaf $\mathcal{O}_{X^m}$.
Your question: "In which sense the schemes generalize the notion of variety and why is better to work with this notion?"
Answer: In general the notion "scheme" is better since it gives a definition valid for arbitrary commutative unital rings. The definition given above was a suggestion in response to a question on MO on the problem of defining algebraic varieties/schemes (including non-reduced schemes) without using prime ideals and where the definition was "more intuitive". With the definition given above it follows a point $x\in X^m$ corresponds to a solution of a system of polynomial equations over a field.
There were several suggestions, and one of these suggestions used "classes". With the definition given above you avoid using classes: You work with the "set of closed points" in a fixed scheme $X$, hence there is no non-standard set theory involved. The idea was that this definition would be easier to understand for students in algebraic geometry.
