Suppose $y \in cl(A)$. If $y\in A$ there isn't anything significant to notice. $d(x,y) \in D= \{d(x,a)|a \in A\}$ and $d(x,y) \ge \inf D = d(x,A)$.
If $y \not \in A$ then $y$ is a limit point of $A$ so for any $\epsilon > 0$ there is a $w \in A$ so that $d(w,y) < \epsilon$ and $d(x,w) \le d(x,y) + d(w,y) $ so $d(x,y) \ge d(x,w) - d(w,y) > d(x,w) -\epsilon \ge d(x,A) -\epsilon$.
So $\epsilon$ is arbitrary if $d(x,y) < d(x,A)$ then we can get a contradiction by fixing $\epsilon = d(x,A) - d(x,y)$ which would contradictorily mean $d(x,y) > d(x,y)$.
So $d(x,y) \ge d(x,A)$ so $d(x,A)$ is a lower bound of $B = \{d(x,b)|b\in cl(A)\}$. So $d(x, cl(A)) \ge d(x, A)$.
We can do the same argument to show the other way around but as you noted, it is obvious that as $A \subset cl(A)$ so $D \subset B$ and so $d(x,A) = \inf D \ge \inf B = d(x, cl(A))$.
So $d(x, A) \ge d(x, cl(A))$ and $d(x,A) = d(x,cl(A))$.
======
...or...
Suppose $d(x,A) > d(x,cl(A)) $.
Then let $y \in d(x,cl(A)$ and let $d(x, cl(A) \le d(x,y) < d(x,A)$. Then $y \not \in A$ so $y$ is a limit point of $A$. Let $\epsilon = d(x,A) - d(x,y)$. Then there is a $w \in A$ so that $d(w,y) < \epsilon$. So $d(x,A) \le d(x,w) \le d(x,y) + d(w,y) < d(x,y) + \epsilon = d(x,A)$ so $d(x,A) < d(x,A)$; a contradiction.
So $d(x,A) \le d(x,cl(A)$.