Quotients of rings not necessarily by ideals. Let $A \subset \Bbb{Z}$ be such that, there is a well-defined map $\pi: \Bbb{Z} \to \Bbb{Z}/A = \{ x + A : x \in \Bbb{Z}\}$ that's a homomorphism of rings.
Thus we're not strictly tied to ideals $A$.
For example, let $A = \pm$ the odd primes.  Then $x + A = y + A \iff x = y$ otherwise $x -y + A = A$ or there is a number $z$ such that $z + p$ is an odd prime for every $p\in A$. Such a number can easily be shown not to exist.  Thus a well-defined multiplication and addition on $\Bbb{Z}/A$ are induced.
Thus the kernel of the map $\pi$ is $0$ and $\Bbb{Z}/A \simeq \Bbb{Z}$.  However, if we enlarge the set $A$ to be the full set of odd numbers.  Then we also get a well defined map $\pi$:
$x + A = y + A \iff x - y + A = A \iff x - y = 2k$ for some $k \in \Bbb{Z}$.  It is an exercise to manually check that multiplication and addition are satisfied.  But we'll do it just to be sure:
$$
\bar{x} = \bar{x}', \bar{y} = \bar{y}' \implies \\
x - x' = 2m, \ y - y' = 2n \implies \\
(x + y) - (x' + y') = 2(m + n) \\ \text{and,}
 \ (x - x')(y - y') = 2mn \iff \\xy - x'y' = x'y -y'x + 2mn = 2k
$$
Thus $\ker \pi = 2\Bbb{Z}$ and $\Bbb{Z}/A \simeq \Bbb{Z}/2\Bbb{Z}$.
Conjecture.  If $A \subset B$ then $\ker \pi_A \subset \ker \pi_B$.
Let $A = $ the odd primes, and $B = $ the odd numbers.  
Question. Can you describe a proper subset $A \subsetneq C \subsetneq B$ such that $Z/C$ is well-defined?
I think this might relate to the problem of $A - A = 2\Bbb{Z}$ which hasn't been proved yet.
 A: Let $R$ be a ring and $A\subseteq R$. Let $R/A = \{x+A\mid x\in R\}$ be the set of cosets. Define relation $\equiv_A$ on $R$ with $$x\equiv_A y\iff x+A = y+A.$$
This relation is obviously equivalence relation on $R$. $R/A$ is a ring with well defined operations $$(x+A)+(y+A) = (x+y)+A\\(x+A)(y+A) = (xy+A)$$
induced by operations on $R$, if and only if $\equiv_A$ is a congruence, meaning that $\equiv_A$ is compatible with operations on $R$. 
Denote with $\operatorname{Con}R$ the set of congruences on $R$ and with $\operatorname{Id}R$ the set of ideals of $R$. Let us define functions
\begin{align}
\varphi\colon \operatorname{Con}R&\to\operatorname{Id}R &\psi\colon \operatorname{Id}R&\to\operatorname{Con}R\\
\equiv\, &\mapsto \{x\in R\mid x\equiv 0\} &I&\mapsto \{(x,y)\in R\times R\mid x - y\in I\}.\end{align}
You can easily verify that these functions are well defined and that they are inverse to each other. Moreover, they are order preserving! If we extend $\psi$ to all subsets of $R$, then it is well known that the induced relation $\psi(I)$ is congruence if and only if $I$ is ideal.
Let us now return to our congruence $\equiv_A$ and its associated ideal $I_A$. It is clear that $x+A = y + A$ if and only if $x-y+A = A$, i.e. $x\equiv_A y$ if and only if $x - y\in I_A$, which means that $\equiv_A$ is equal to induced congruence by $I_A$. We conclude that $\equiv_A$ is congruence if and only if $I_A$ is ideal.
If $R/I_A$ is standard quotient by ideal, you can immediately verify that $R/I_A\cong R/A$, with isomorphism defined as $x+I\mapsto x+A$.
Returning to your examples. 


*

*Let $A = \{\pm p\mid p\ \text{odd prime}\}$. Instead of checking that $\equiv_A$ is congruence, by the above discussion it is enough to check that $I_A = \{x\in R\mid x + A = A\}$ is ideal. Indeed, we get that $I_A = \{0\}$, so $\mathbb Z/A$ is a ring isomorphic to $\mathbb Z$.

*Let $B = \{2n+1\mid n\in\mathbb Z\}$. Then $I_B = 2\mathbb Z$, so $\mathbb Z/B$ is ring isomorphic to $\mathbb Z/2\mathbb Z$.

So far everything looks fine and dandy. However, working with general subsets of $R$ instead of ideals is very ill-behaved. The obvious reason is that we don't even know if $R/A$ is a ring or not. Furthermore, when $I\subseteq J$ are ideals of $R$, we get ring epimorphism $R/I\to R/J$. When $A\subseteq B$ are subsets of $R$, there might not even be nontrivial homomorphism $R/A\to R/B$.
For example, let $I$ a nonzero, proper ideal and $A = I\cup\{a\}$, $a\not\in I$. To prove that $R/A$ is a ring, we need to prove that $I_A = \{ x\in R\mid x + A = A\}$ is an ideal.
First, let me show that $I_A \subseteq I$. Let $x\in I_A$ and assume that $x\not\in I$. For all $y\in I$, we have $x + y \not\in I$ and in particular, $x + y = a$, i.e. $x = a - y$. This is contradiction, since $I$ is nonzero. Thus, $x\in I$.
Now, let $x\in I_A$. Since $a\not\in I$ and $x\in I$, we have $x+a\not\in I$, so $x+a = a$. Thus, $x = 0$. We conclude that $I_A = \{0\}$. In particular, $R/A$ is a ring isomorphic to $R$. Also, this example gives negative answer to your conjecture.
To answer your Question, let $C = \{2n+1\mid n\in\mathbb Z\}\setminus\{1\}$. To determine $I_C$, first note that $I_C \subseteq 2\mathbb Z$. Assume that $x\in I_C$, $x\neq 0$. Then, $x = 2n$, for nonzero $n$. Define $y = 1 - 2n\in C$. Then we have $x + y = 1\not\in C$. We conclude that $I_C = \{0\}$ and in particular $\mathbb Z/C$ is a ring isomorphic to $\mathbb Z$.

Finally, let me give an example of set $A$ such that $R/A$ is not a ring. If $A$ is additive subgroup of $R$, then it is easy to check that $I_A = A$. Thus, by the above general argument, $R/A$ is a ring if and only if $A$ is an ideal of $R$. So find additive subgroup that is not an ideal, and you will get an example of $R/A$ not being a ring.
However, if $A$ is additive subgroup, $R/A$ is abelian group. This is because, analogous to rings, there is bijective correspondence between congruences of a group and its normal subgroups.
