In any metric space, all compact sets are closed. To see this, let $(X,d)$ be a metric space, and $Y \subset X$ a non-closed set. Since $Y$ is not closed, it does not contain all of its limit points, so there exists a point $y \not\in Y$ which is an accumulation point of $Y$. Then, the collection
$$\mathscr{U} = \{U_\varepsilon\}_{\varepsilon > 0},$$
where $U_\varepsilon = \{x \in X : d(x,y) > \varepsilon\}$, is an open cover of $Y$. However, any finite subset of $\mathscr{U}$, for example
$$\mathscr{V} = \{U_{\varepsilon_1}, \cdots, U_{\varepsilon_n}\}$$
is not a cover of $Y$. To see this, let $\varepsilon = \min\limits_{i\in\{1, \cdots, n\}}\varepsilon_i$. Then, the ball around $y$ of radius $\varepsilon$ is disjoint from every set in $\mathscr{V}$ and contains a point of $Y$ (since $y$ was assumed to be a limit point of $Y$). It follows that $Y$ is not compact.
As Prism mentioned in the comments, we can show that any compact subset of a Hausdorff space (a class of topological spaces which includes metric spaces) must be closed. So we have to look for a non-Hausdorff (non-metric) counterexample. One such example is the set $X = \{x_1, x_2, x_3\}$ with the trivial topology $\tau = \{\emptyset, X\}$, where the set $Y = \{x_1\}$ is compact (since any open cover of $Y$ is finite), but $Y$ is not closed.