Having fun with arithmetic. Can I cover a circle with irrationals using unary operation? Got some time to have a fun with arithmetic, but quickly went beyond the picture I can imagine and explain to myself. Here would like to ask you what mathematical structures I deal with and if it is possible to continue my thoughts one more step further.

*

*The object I work with is a circle (like in geometry) with a "starting" point on it.

*There is an unary operation "next" which applies to a point and gives other "next after starting" point on a circle.

*Repeating the operation on it, gives one more "next after next after starting" point on a circle.

*After some number of iterations it loops back to a "starting" point.

Case 1.

*

*Circumference is 17.

*Starting point is $0$.

*Operation is increment by 1 mod 17.

Repeating the operation constructs a set of integers {0..16}.
Case 2.

*

*Circumference is 17.

*Starting point is $1$.

*Operation picks next element in a somewhat reduced Stern-Brocot tree. Explanation follows.

Cannot post images, I refer to this picture of Stern-Brocot tree: https://mathworld.wolfram.com/images/eps-svg/SternBrocotTree_1000.svg
Reduced Stern-Brocot tree is a Stern-Brocot tree with right branches after 17 cut-off.
Next element is a right neighbor on the same tree level (or leftmost one on next level). For instance, "next after 2/3" is 3/2 for it is on the right on the same level, while "next after 3/1" is 1/4.
Repeating the operation, covers circle with rational numbers in a continuous range (0, 17).
But I also need to loop it back to a "starting" point $1$. Doubt if it makes sense, but I feel like I can go with infinite ordinal $\omega$ iterations and slightly modify the operation which picks "next" point in such a way, that $(\omega + n)$th point is the same as $(n - 1)$th point on my circle. Thus it mimics "mod 17" from case 1.
Finally the problem.
Can I define some unary operation on the circle to cover it continuously with irrational or real numbers from a range (0, 17) so that it loops back after some number of iterations?
 A: The question has several subtle points we need to touch to give a full picture.
First of all the idea of circling, in your first example you had a finite set of points, and hence you could have a simple cycle, in your second case you had a process going through every rational point and then you wanted to "go back".
As you correctly stated, one need to go to the ordinals after countably many steps. Your process was defined recursively on all ordinals:
For $a_0$ you took $1$, for $a_{n+1}$ is the next element in the reduced tree after $a_n$, and for $α$ being limit ordinal you let $a_α=1$.
This is how any transfinite construction would (usually) look, you define the step "$0$", the step "$α+1$", and the step "$β$" for limit ordinals $β$.

The second point we need to talk about is cardinalities, the first example had a set of points with finite cardinality, your second example had a set of points with countable cardinality, you are asking for covering of a set of points with continuum cardinality, indeed the cardinality of the reals between $0$ and $17$ is uncountable.
This means that whatever process we would ever hope to get won't be "as nice" as the 2 examples you gave, in particular the step $ω$ won't be the same as the first step.
It doesn't mean we can't achieve anything, it just means that our cycles will be longer, similarly to how the cycle of the second example wasn't finite.

Thirdly the cycle itself.
Here the more complicated ideas start, I stated before that we describe processes using ordinals, under this premise a cycle is just a well ordering of the set $(0,17)$.
That means that we give a new order $\prec$ on $(0,17)$ (which is not the usual order on the real numbers) such that if $A\subseteq (0,17)$ then there is an element in $A$ that is minimum with respect to $\prec$.
It turns out that every well ordering is isomorphic to a unique ordinal number, so given a well ordering $\prec$ on $(0,17)$ we define the following "circle" (I will assume that $\prec$ has ordertype exactly $|(0,17)|$, that means that if $x\in (0,17)$ then $|\{y\in (0,17)\mid y\prec x\}|<|(0,17)|$, this will make the description easier. One can show that if a well ordering exists, then there exists a well ordering with the property I said above):
At the steps of the form $x={\frak c}^{β}\cdot α+γ+δ$ where $|(0,17)|=\frak c$, and $α<{\frak c},β,{\frak c}<γ<{\frak c}^{β},δ<{\frak c}$ are ordinals (every ordinal can be written uniquely in this form, this is a variation of Cantor's normal form) let $a_x$ be the unique $r\in (0,17)$ such that $\{t\in(0,17)\mid t\prec r\}$ has the ordertype as $δ$.
The existence of such ordering for the set $(0,17)$ is not trivial, it follows from the axiom of choice which is equivalent to the well-ordering theorem which states that "for every set there exists a well ordering on that set" (not to be confused with the well-ordering principle, which states that the set of natural numbers with the usual order is well ordered).
The classical theory mathematicians work in is ZFC, "Zermelo-Fraenkel set theory with the axiom of Choice".
If one remove the axiom of choice and assumes only ZF ("Zermelo-Fraenkel set theory without the axiom of choice") it is possible that there is no way to order the set $(0,17)$ using well ordering, so in ZF it is possible that there is no way to achieve what you are asking for.

The last point we will talk about is the "process" part, in your 2 examples you didn't simply said "there is a way to get a cycle", you actually defined the method we pick the next element.
Having an explicit way is stronger than just having a well ordering.
Definability is one of the most confusing concepts to beginners/amateur mathematicians so I don't want to go into details about this point, but for the purpose of this question we say that an order $\prec$ is definable if there is a formula $φ(x,y)$ such that $a\prec b$ if and only if $φ(a,b)$. (this "definition" is not a real definition, but it will be enough for us).
Now in the last part I said that it is possible to have a well-ordering on $(0,17)$, the question now is: can our well-ordering be definable?
The answer for this is: yes, it is possible, but it is also possible that there is no definable well ordering.
If this sounds weird to you, it is okay, definability is weird. We have a formula $φ(x,y)$ such that we know that $φ$ defines a well ordering on some set $X$, but we don't know if $(0,17)⊆X$! (we have a lot of such $φ$, for an explicit example see this, this is a well ordering on $L$, a class that may contains all of the real numbers, but it also may miss a lot of real numbers)
To show that it is possible to not have any definable well ordering you need a much more advance method called forcing, you can see here a proof that shows that if $φ(x,y)$ is a formula, then ZFC cannot prove that $φ(x,y)$ is a well-ordering of the real numbers.
That is: It is independent from ZFC (the standard axioms of mathematics) that there exists an explicit process that defines a circle over the numbers $(0,17)$
