A noetherian local ring $(A,\mathfrak m,k)$ is regular if
$dim(A)=dim_k\mathfrak m / \mathfrak m^2$. The left-hand side refers of course to Krull dimension and the number on the right-hand side is also the smallest numbr of generators of $\mathfrak m$ (by Nakayama's lemma).
In dimension zero a local noetherian ring is regular iff it is a field. So for example $k[T]/(T^2)$ is a noetherian non regular local ring of dimension zero.
In dimension one a noetherian local ring is regular iff it is a PID, just as you said.
In higher dimension regularity is to be thought as being manifold-like and is related to (but not identical with) smoothness.
Finally a noetherian ring $R$ is said to be regular if its localizations $R_\mathfrak m$ at all maximal ideals $\mathfrak m$ are regular in the sense above.
Let's change the subject.
The projective dimension $pd(M)$ of an $R$-module $M$ is the smallest integer $r$ ( or $\infty) $ such that there exists a resolution $0 \to P_r \to \ldots \to P_1 \to P_0 \to M\to 0 $ of length $r$ with all $P_i$'s projective $R$-modules.
The global (or homological) dimension $gldim(R)$ of $R$ is the supremum of the projective dimensions $pd(M)$ over all $R$-modules $M$ (it is an integer or $\infty$) .
And now for the surprise: I haven't really changed the subject! Serre, Auslander and Buchsbaum proved that a local noetherian ring is regular if and only if it has finite global dimension.
The equivalences you require then follow quickly :
$$A \; regular \iff A[X] \; regular $$
$$A \; regular \iff A[[X]] \; regular $$
For a proof see Bruns-Herzog, Cohen-Macaulay rings, 2.2.13 page 69.