How to find the class number of $\mathbb{Q}(\sqrt{-17})$?

I tried to calculate the class number with help of the Minkowski bound of $M \approx 5$. So if an ideal has norm $1$, it is the ring of integers. If it has norm $2$, it is $(2, 1+\sqrt{-17})$, which is not principal. Norm 3 gives us the ideals $(3, 1+\sqrt{-17})$ and $(3, 2+\sqrt{-17})$, which are both not principal. The only ideal of norm 4 is the ideal (2) and there arent any of norm 5.

I don't know how to find the class number given this information. Can you guys please help?

From your calculation, you know the following facts about the class number $h = h_{\mathbb{Q}(\sqrt{-17})}$ already

1. $h > 1$, because $\mathfrak{p}_2 = (2, 1 + \sqrt{-17})$ is a non-principal ideal

2. $h \leq 5$, because you have found 5 ideals with norm less than the Minkowski bound.

We have $h = 2, 3, 4, \text{ or } 5$. But to pin down $h$, you need to do more work...

Step 1: From your computation of $\mathfrak{p}_2 = (2, 1 + \sqrt{-17})$ being an ideal of norm 2, you probably have found that $(2) = \mathfrak{p}_2^2$ since $-17 \equiv 3 \pmod{4}$. Or just directly check this by computing the product $\mathfrak{p}_2^2$.

Because $(2)$ is principal, this shows that in the class group, $[\mathfrak{p}_2]$ is an element of order 2. But what do you know about the order of an element in a finite group? It must divide the order of the group. So which possibilities for $h$ can we now eliminate?

Step 2: We can ask a similar question about $\mathfrak{p}_3 = (3, 1 + \sqrt{-17})$: is $\mathfrak{p}_3^2$ principal? If it was, then it would be an ideal of norm $3 \times 3 = 9$. So it would be generated by an element $\alpha$ with norm $\pm 9$. But solving $N(\alpha) = x^2 + 17y^2 = 9$ shows $\alpha = \pm 3$. But when you have factored $(3)$ to find $(3) = \mathfrak{p}_3 \widetilde{\mathfrak{p}_3}$, you can also say $\mathfrak{p}_3 \neq \widetilde{\mathfrak{p}_3}$. This means that by the unique factorisation of ideals in number rings, $\mathfrak{p}_3^2 \neq (3)$, so it cannot be principal.

(Alternatively: compute that $\mathfrak{p}_3^2 = (9, 1 + \sqrt{-17})$. If this is going to equal $(3)$, then we must have $1 + \sqrt{-17} \in (3)$, but clearly $3 \nmid 1 + \sqrt{-17}$, so this is not possible. Again conclude $\mathfrak{p}_3^2$ is not principal.)

This means that in the class group $[\mathfrak{p}_3]$ has order $> 2$. This shows that $h > 2$ because the order of an element must divide the order of the group.

Conclusions: You have enough information now to conclude that $h = 4$, agreeing with Will Jagy's answer. Step 1 shows that $h = 2, 4$, and step 2 shows that $h = 4$, so we're done.

In fact we also know the structure of the class group $\mathcal{C}(\mathbb{Q}(\sqrt{-17}))$. It is a group of order $h = 4$, so either $\mathcal{C}(\mathbb{Q}(\sqrt{-17})) \cong \mathbb{Z}_2 \times \mathbb{Z}_2 \text{ or } \mathbb{Z}_4$. But since $\mathfrak{p}_3$ has order $> 2$, it must have order 4. This shows that $\mathcal{C}(\mathbb{Q}(\sqrt{-17}) \cong \mathbb{Z}_4$.

I get $4.$ Maybe someone will describe that in language you find suitable. For positive forms (imaginary fields), the number of classes of forms, same as number of reduced forms of the discriminant $-68,$ agrees with your calculation.

 jagy@phobeusjunior:~/old drive/home/jagy/Cplusplus\$ ./classGroup
Absolute value of discriminant?
68
Discr  -68 = 2^2 * 17  class  number  4

all
68:  < 1, 0, 17>    Square        68:  < 1, 0, 17>
68:  < 2, 2, 9>    Square        68:  < 1, 0, 17>
68:  < 3, -2, 6>    Square        68:  < 2, 2, 9>
68:  < 3, 2, 6>    Square        68:  < 2, 2, 9>