What is an example of a Dedekind ring that is non-principal? Prop. 15 of Serge Lang's ANT shows that a sufficient condition for a Dedekind ring $R$ to be principle is that it only have finitely many primes.
To give an outline of the argument, one starts with a factorization of any ideal $\mathfrak a = \prod_1^n \mathfrak {p}_i^{r_i}$ and proceeds to pick elements $\pi_i \in \mathfrak p_i \setminus \mathfrak p_i^{2}$. Using the Chinese remainder theorem, one can then find $\alpha \equiv \pi_i^{r_i}\pmod {\mathfrak p_i^{r_i+1}}$. Writing down the factorization of $(\alpha)$ into prime ideals $(\alpha) = \prod_1^n \mathfrak {p}_i^{e_i}$ we immediately get that $e_i = r_i$ and hence $\mathfrak a = (\alpha)$.
As I understand the proof, the assumption of $R$ having finitely many primes is critically used at the step where we pick $\alpha$ because we ensure it does not lie in any prime other than those in the factorization of $\mathfrak a$. But how do you come up with a Dedekind domain $R'$ and an ideal $\mathfrak a'$ such that no matter what $\alpha'$  you pick satisfying the congruence relations as above, it will always intersect some prime not in the factorization of $\mathfrak a'$?
 A: Georges has given you a good geometric example. Just so this is answered, let me say some words.
Yes, as you point out the key point is being able to control the factorization of $(\alpha)$ so precisely. 
Let us explain this a little more. In what follows, let $R$ be a Dedekind domain.
Now, remember that ideals $I$ of $R$ are completely in correspondence with the set $X:=\mathbb{N}^{\oplus\text{MaxSpec}(R)}$ of tuples $(e_\mathfrak{p})_{\mathfrak{p}\in\text{MaxSpec}(R)}$ and $e_\mathfrak{p}=0$ for almost all $\mathfrak{p}$. The correspondence, of course, is 
$$F:\displaystyle(e_{\mathfrak{p}})\mapsto \prod_{\mathfrak{p}}\mathfrak{p}^{e_\mathfrak{p}}$$
This is a slightly less highfalutin way of saying that the monoid of ideals of $R$ is just the free abelian monoid on $\text{MaxSpec}(R)$. Or, because we're more common with groups than monoids, this is usually stated that the group of fractional ideals of $R$ is the free abelian group on $\text{MaxSpec}(R)$.
Now, let $\mathcal{P}\subseteq X$ be the set of tuples $(e_\mathfrak{p})$ such that $F((e_\mathfrak{p}))$ is principal. In other words, it's the set of tuples in $X$ corresponding to principal ideals of $R$. The statement that $R$ is a PID is then just that $\mathcal{P}=X$.
Now, with this language, there is a very nice way to think about the Chinese Remainder Theorem. It basically says that if you pick finitely many indices $\mathfrak{p}_1,\ldots,\mathfrak{p}_n\in\text{MaxSpec}(R)$ and integers $e_1,\ldots,e_n$ you can always find some tuple $(e_\mathfrak{p})\in\mathcal{P}$ such that $e_{\mathfrak{p}_i}=e_i$. In other words, if you only care about finitely many coordinates of a tuple $(e_\mathfrak{p})$, and are not concerned with the other coordinates, you can always find a tuple coming from a principal ideal that fits your needs. 
Of course, the above does NOT say that for any set of primes $\{\mathfrak{p}_i\}$ and any set of integers $\{e_i\}$ you can find an element $(e_\mathfrak{p})\in\mathcal{P}$ such that $e_{\mathfrak{p}_i}=e_i$. Indeed, this says precisely that $\mathcal{P}=X$ (taking the set $\{\mathfrak{p}_i\}$ to be $\text{MaxSpec}(R)$). The issue is that, when we were discussing the Chinese Remainder Theorem, we could only specify finitely (but arbitrarily large) many places, and have the others out of our control.
But, if we are in the case where $\text{MaxSpec}(R)$ is finite, then we see that the last paragraph's warning is actually moot. Namely, then we really can specify, for any $(e_\mathfrak{p})\in X$ a point $(f_\mathfrak{p})\in\mathcal{P}$ such that $e_\mathfrak{p}=f_\mathfrak{p}$, or, in other words, $X=\mathcal{P}$ which, as we have noted before, is just saying that $R$ is a PID.
But, to show you that we're not missing something, that there isn't some stronger version of the Chinese Remainder Theorem that we are missing, that really would allow the cautionary paragraph to ALWAYS be moot (i.e. not just moot when $\text{MaxSpec}(R)$ is finite) we just need to produce a non-PID Dedekind domain. Georges has provided you with one, but since this is the algebraic number theory category, perhaps a more arithmetic example will help. 
Let's show that $A:=\mathbb{Z}[\sqrt{-5}]$ is not a PID. Indeed, let us take the ideal $\mathfrak{p}=(2,1+\sqrt{-5})$. To see that $\mathfrak{p}$ is not principal we merely check that 
$$\begin{aligned}A/\mathfrak{p}&\cong \mathbb{Z}[x]/(x^2+5,2,1+x)\\ &\cong \mathbb{F}_2[x]/(x^2+5,1+x)\\ &\cong \mathbb{F}_2[x]/((x+1)^2,1+x)\\ &\cong \mathbb{F}_2[x]/(x+1)\\ &\cong \mathbb{F}_2\end{aligned}$$
Thus, in particular $[A:\mathfrak{p}]=2$. But, I leave it to you (using similar techniques to the above) to show that $[A:(a+b\sqrt{-5})]=a^2+5b^2$. Thus, if $\mathfrak{p}$ was principal, say $\mathfrak{p}=(a+b\sqrt{-5})$ then 
$$2=[A:\mathfrak{p}]=[A:(a+b\sqrt{-5})]=a^2+5b^2$$
But, since $a,b\in\mathbb{Z}$ this is clearly impossible, and so $\mathfrak{p}$ really isn't principal. So, $A$ is an example of a Dedekind domain which is not a PID.
An interesting question one might ask, especially with the above discussion, is if we can measure precisely how far $R$ is from being a PID. In fact, the answer is yes. Namely, in the above we noticed that $X$ was a monoid, and a little thought shows that $\mathcal{P}$ defines a congruence inside of $X$. Thus, we can form the quotient monoid $X/\mathcal{P}$ which precisely measures how badly $\mathcal{P}=X$ fails, and thus how badly $R$ fails to be a PID. This monoid $X/\mathcal{P}$ turns out to actually be a group, called the class group of $R$, it is often denoted $\text{Cl}(R)$. It plays a pivotal role in algebraic number theory, and parts of algebraic geometry, as you can probably deduce. 
For example, $\text{Cl}(A)=\mathbb{Z}/2\mathbb{Z}$, and so while $A$ is not a PID, it's failure to be so is, in a sense, as minimal as possible. 
