It's hard to get an intuitive handle on compactness at first. One way to view a compact set is as one that behaves, topologically, as if it were finite. I don't know if that actually helps, but I'll try to explain it anyway.
Certainly every finite set is compact, no matter the topology. (You should prove this.) And it is possible that finite sets are the only compact sets, if the topology is right. (You should prove this too.) So the claim that compactness is something like finiteness is not completely insane.
But let's consider an infinite compact set, say $[0,1]$ in the usual topology. Since we are doing topology, we would like to understand points by what open sets they are in. So for each point $p$ of $[0,1]$, pick an open set $G_p$ containing $p$, and let $G$ be the collection of all $G_p$. The family $G$ is an uncountable collection of sets! But because $[0,1]$ is compact, we can find a simpler way to understand $[0,1]$. There is a finite subset $F\subset [0,1]$ for which each point of $[0,1]$ is in $G_p$ for some $p$ in the finite subset $F$. If $x\in G_p$ means that $x$ is ‘close to’ $p$, then every point of $[0,1]$ is ‘close to’ some element of the finite set $F$. And no matter how complicated the original family $G_p$ is, we can always simplify it to a finite family of the $G_p$ and every element of $[0,1]$ will be in one of the members of the simpler family.
I would like to add that your claim that you can understand why $[0,1]$ is compact is suspect, because your explanation of why $[0,1]$ is compact is missing the important part. Obviously $[0,1]$ can be covered by $(0-\epsilon, 1+\epsilon)$. That is not the reason that $[0,1]$ is compact. Every set can be covered by a finite family of open sets (you should prove this), so this property is not interesting. The “subfamily” part of compactness is crucial and you can't leave it out. But your explanation did leave it out. (This is a very common misunderstanding among people who are new to compactness.)
To show compactness of some set $S$, it is not enough to show that $S$ is covered by a finite family of open sets; this is trivial, because every set, compact or not, is covered by a finite family of open sets. Instead, the proof must go like this:
- An adversary gives you $G$, which is a family of open sets for which $S\subset \bigcup G$
- You find a finite subset $G'\subset G$ for which $S\subset \bigcup G'$.
If you can guarantee to succeed in step 2 regardless of which $G$ was given to you in step 1, you win, and $S$ is compact. Conversely, if you want to show that $S$ is not compact, you and the adversary switch roles: you produce a family $G$ in step 1, and try to foil the adversary, who plays step 2. You win if the adversary cannot find a finite $G'\subset G$ for which $S\subset\bigcup G'$.
I suggest you try this:
- Show that $(0,1)$ is not compact: find the step 1 family $G$ of open sets so that your adversary can't win in step 2.
- Show that $[0,1]$ is compact: let the adversary give you a family $G$ and show how to find finite $G'\subset G$ so that $G'$ still covers $[0,1]$.
If you can do those two basic exercises, you will have made a good start on understanding compactness.