There are formal ways to define dimension, indeed, lots of them, depending on the context. As already noted in a comment, your description of why a circle is one-dimensional is not really correct though. The one-dimensionality of a circle is not a function of the fact that a circle itself can be described by a single number (such as radius), but that to describe the points on the circle takes a single number --- e.g. the usual parameterization $(\cos\theta,\sin\theta)$ of a circle requires just one number, the angle $\theta$. A sphere has two-dimensions because it takes two parameters to describe a point on the sphere (e.g. latitude and longitude). Of course, the sphere itself is again determined by one number, its radius (but that doesn't make it one-dimensional).
To make this more precise, one has to use ideas from topology, as mentioned in
Fredrik Meyer's answer. A key point is that the map we use to parameterize the circle/sphere/whatever should be continuous (in fact even more; at least in a neighbourhood of each point it should also admit a continuous inverse); the kind of bijection you indicate between points on the $(x,y)$-plane and points on the line will not be continuous).
There are other ways to define dimension that don't use parameters, but use different topological properties.
Of these, perhaps the notion of covering dimension is the most basic. To get the idea of it, think first about a ruler: we can divide it up into inch markings, and at any point there will be at most two different inch-long intervals meeting each other. Now think about a brick wall: if the brick layer did a bad job (just stacking bricks on top of one another in columns) then there will be points where four different bricks are in contact, but even if they lay the bricks properly, there will be points where three different bricks are in contact (this is what you usually see in a brick wall, when a single brick in one layer will sit on top of two bricks in the layer below); you can't avoid having points where three bricks meet. (Here I am ignoring the mortar in between the bricks.)
Now think about making a solid pyramid out of stone blocks: even if you arrange the blocks as stably as possible, you will have points where four stone blocks meet.
So: paving a line (one-dimensional) forces some points to have to paving stones meeting; paving a plane (two-dimensional) forces some points to have three paving stones meeting; paving a solid (three-dimensional) forces some points to have four paving stones meeting; you can probably see the pattern!
This is a more topological, less analytical, definition of dimension, which is
important in theoretical investigations of dimension. (For example, it underlies the Cech definition of cohomology in topology.)
There is also the notion of Hausdorff dimension, which captures the idea of dimension in terms of the amount of volume that a figure occupies. It can give non-integral values of dimension, and so is used for discussing the dimension of fractals.