Every infinite subset of $E$ in $\mathbb R^k$ having a limit point in $E$ implies $E$ is closed This is exactly what is written in Walter Rudin chapter 2, Theorem 2.41:
If $E$ is not closed, then there is a point $\mathbf{x}_o \in \mathbb{R}^k$ which is a limit point of $E$ but not a point of $E$. For $n = 1,2,3, \dots $ there are points $\mathbf{x}_n \in E$ such that $|\mathbf{x}_n-\mathbf{x}_o| < \frac{1}{n}$. Let $S$ be the set of these points $\mathbf{x}_n$. Then $S$ is infinite (otherwise $|\mathbf{x}_n-\mathbf{x}_o|$ would have a constant positive value, for infinitely many $n$), $S$ has  $\mathbf{x}_o$ as a limit point, and $S$ has no other limit point in $\mathbb{R}^k$. For if $\mathbf{y} \in \mathbb{R}^k, \mathbf{y} \neq \mathbf{x}_o$, then
\begin{align}
|\mathbf{x}_n-\mathbf{y}| \geq{} &|\mathbf{x}_o-\mathbf{y}| - |\mathbf{x}_n-\mathbf{x}_o|\\
\geq {} & |\mathbf{x}_o-\mathbf{y}| - \dfrac{1}{n} \geq \dfrac{1}{2} |\mathbf{x}_o-\mathbf{y}| 
\end{align}
for all but finitely many $n$. This shows that $\mathbf{y}$ is not a limit point of $S$.
The question:
I'm stuck in understanding the reason behind why $S$ is infinite. 
Also I need clarification why the last inequality holds. May someone help, please?
 A: Suppose $S$ is finite. Then the set $\{\, |\mathbf x - \mathbf x_o| : \mathbf x \in S \,\}$ is a finite set of strictly positive numbers. Thus it has a smallest member, which is a positive number. Choose $m$ so large that $1/m$ is less than that smallest number. Then no points of $E$ are within a distance $1/m$ of $\mathbf x_o,$ so $\mathbf x_o$ would not be a limit point.
Let's parse the part about having "constant positive value for infinitely many $n.$"
If a set is finite, then a sequence of members of that set has "constant positive value for infinitely many" terms. For example, consider the digits $0,1,2,3,4,5,6,7,8,9.$ The decimal expansion of an irrational number cannot contain only finitely many of each of those.
If a particular positive number occurs infinitely many times in a sequence, then the sequence cannot approach $0.$ But the sequence of distances $( |\mathbf x_n - \mathbf x_o| )_{n=1}^\infty$ approaches $0.$
That last inequality holds if $n$ is large enough, since in that case $1/n$ is small.
A: Suppose $S$ is finite, say $S=\{y_1,\cdots,y_k\}\subseteq\{x_n:n \in \mathbb N\}\subseteq E.$ Then the possible values each $|x_n-x_0|$ can take are $$|y_1-x_0|,\cdots,|y_k-x_0|.$$
So there will exists $i \in \{1,\cdots,k\}$ such that $|x_n-x_0|=|y_i-x_0|$ for infinitely many $n.$ So we have $$|y_i-x_0|\leq \frac 1n$$ for infinitely many $n.$ Thus, $x_0 = y_i \in E,$ which is a contradiction.

Since, $x_0 \neq y,$ then there exists $n_0 \in \mathbb N$ such that $$|x_0-y|>\frac {2}{n_0}>\frac 2n$$ for all $n \geq n_0.$ Thus,
$$\frac 12 |x_0 -y|>\frac 1n$$ for all $n \geq n_0.$
So, $$|x_0-y|-\frac 12 |x_0-y|>\frac 1n$$ for all $n\geq n_0.$ Hence, the last inequality follows.
A: If $S$ was finite, then you can take minimum of $|x_n - x_0|$, since there are finitely many $x_n\in S$. (minimum is not $0$ since $x_0\notin S$) But this contradicts with the fact that there are $x_n$'s satisfying $|x_n-x_0| < 1/n$ for all $n$.
For the last inequality;
\begin{equation}
|\mathbf{x}_o-\mathbf{y}| - \dfrac{1}{n} \geq \dfrac{1}{2}|\mathbf{x}_o-\mathbf{y}|
 \iff \dfrac{1}{2}|\mathbf{x}_o-\mathbf{y}| \geq \dfrac{1}{n}
\end{equation}
and this is true for all but finitely many $n$, since $\dfrac{1}{2}|\mathbf{x}_o-\mathbf{y}|$ is a fixed positive number and $1/n\to 0$.
A: We say that $\mathbf{x}_o$ is a limit point of $E \subseteq \mathbb{R}^k$ if and only if for every $\varepsilon>0$ there exists $\mathbf{x}_{\varepsilon} \in E$ such that $\mathbf{x}_{\varepsilon} \in B(\mathbf{x}_o,\varepsilon) \setminus \{\mathbf{x}_o\}$.
A set $E \subseteq \mathbb{R}^k$ is closed if and only if it contains all of its limit points. 
Suppose $E \subset \mathbb{R}^k$ is not closed, so there is $\mathbf{x}_o \in \mathbb{R}^k$ such that $\mathbf{x}_o$ is a limit point of $E$ but $\mathbf{x}_o \notin E$. Since $\mathbf{x}_o$ is a limit point of $E$, for $n=1,2,3,\ldots$ there exist $\mathbf{x}_n \in E$ such that $\mathbf{x}_{n} \in B(\mathbf{x}_o,n^{-1}) \setminus \{\mathbf{x}_o\}$. Now the range of the sequence $\{\mathbf{x}_{n}\}_{n=1}^\infty$ is an infinite set; otherwise there is $\mathbf{x}_{N} \in \{\mathbf{x}_{n} : n \in \mathbb{N}\}$ such that $d(\mathbf{x}_{N},\mathbf{x}_{o})<\frac{1}{n}$ for infinitely many $n \in \mathbb{N}$  which would imply that $\mathbf{x}_{N}=\mathbf{x}_{o}$, a contradiction.

Suppose $\mathbf{y} \in \mathbb{R}^k$ and $\mathbf{y}\neq \mathbf{x}_o$. Since $\mathbf{y} \neq \mathbf{x}_o$, it follows from the definition of a metric space that $d(\mathbf{x}_o,\mathbf{y})>0$. By the Archimedean Principle we may find a positive integer $N$ such that $Nd(\mathbf{x}_o,\mathbf{y})>2.$ So if $n\geq N$, then 
\begin{aligned}d(\mathbf{x}_n,\mathbf{y}) & \geq d(\mathbf{x}_o,\mathbf{y}) - d(\mathbf{x}_o, \mathbf{x}_n) 
\\& > d(\mathbf{x}_o,\mathbf{y}) - \frac{1}{n}
\\& \geq d(\mathbf{x}_o,\mathbf{y}) - \frac{1}{N}
\\& > d(\mathbf{x}_o,\mathbf{y}) - \frac{1}{2}d(\mathbf{x}_o,\mathbf{y})=\frac{1}{2}d(\mathbf{x}_o,\mathbf{y}) .
\end{aligned}
Let $F \subset S$ denote the range of the finite sequence $\{\mathbf{x}_j \}_{j=1}^{N-1}$. Now we either have $\mathbf{y} \in F$ or $\mathbf{y} \notin F$, so we consider the cases.
Case $1$:  $\mathbf{y} \in F$. Let $T$ denote the set $T:=\{\mathbf{x} \in F:0<d(\mathbf{x},\mathbf{y})\leq\frac{1}{2}d(\mathbf{x}_o,\mathbf{y})\}$. If $T = \emptyset$ select $\hat{\varepsilon}=\frac{1}{2}d(\mathbf{x}_o,\mathbf{y})$, otherwise select $\hat{\varepsilon}=\min\limits_{\mathbf{x}\in T} d(\mathbf{x},\mathbf{y})$.
Case $2$:  $\mathbf{y} \notin F$. We shall select $\hat{\varepsilon}=\min\{d(\mathbf{x}_1,\mathbf{y}), \ldots , d(\mathbf{x}_{N-1},\mathbf{y}), \frac{1}{2}d(\mathbf{x}_o,\mathbf{y})\}$. 
In either case we have found a number $\hat{\varepsilon}>0$ such that $B(\mathbf{y}, \hat{\varepsilon})\setminus \{\mathbf{y}\} \bigcap S = \emptyset$ which is the negation of the statement "$\mathbf{y}$ is a limit point of $S$".
