What is the algebraic tangent cone really? Let $A$ be a (commutative unital) ring, let $\mathfrak{a} \subseteq A$ be an ideal, and let $B = A / \mathfrak{a}$. Then we have a descending filtration
$$\cdots \subseteq \mathfrak{a}^3 \subseteq \mathfrak{a}^2 \subseteq \mathfrak{a} \subseteq A$$
and the associated graded module
$$C = B \oplus \mathfrak{a} / \mathfrak{a}^2 \oplus \mathfrak{a}^2 / \mathfrak{a}^3 \oplus \cdots$$
is a graded $B$-algebra in an evident way. In the case where $A$ is a local ring and $I$ is the unique maximal ideal, this is the definition of the tangent cone. What about the general case – does the construction still have any geometric significance?
Here are some special cases:


*

*Suppose $\mathfrak{a} = 0$. Then $C \cong B$ (as $B$-algebras).

*Suppose $A$ is an integral domain and $\mathfrak{a}$ is a non-zero principal ideal. Then each $\mathfrak{a}^n$ is also a non-zero principal ideal, and it is not hard to see that $C$ is isomorphic (as a $B$-algebra) to the polynomial algebra $B [t]$.

*Suppose $A \to B$ admits a section (as a ring homomorphism), so that $A$ is a $B$-algebra and $A \cong B \oplus \mathfrak{a}$ as $B$-modules. Then $a \mapsto 1 \otimes \mathrm{d} a$ defines an isomorphism $\mathfrak{a} / \mathfrak{a}^2 \to B \otimes_A \Omega^1_{A \mid B}$, therefore $C$ is a quotient of the symmetric $B$-algebra on $B \otimes_A \Omega^1_{A \mid B}$.


Thus, it seems to me that what we get is some kind of relative tangent cone.
 A: As mentioned in the comment, in general, it is called the $\textit{the associated graded ring}$ of $A$ with respect to $I$. This object has been studied intensively in both commutative algebra and algebraic geometry. 
Write $\operatorname{gr}_I(A)$ for the $C$ above. For simplicity, let's also assume that $(A,m, k = A/m)$ is a Noetherian local ring. 
Let $I$ be an ideal which is primary to the maximal ideal $m$, so that the length of $R/I^i$ is finite for all $i$. Then one can define the function
$$
H_A(i): \mathbb{N} \to \mathbb{N}
$$
as $H_A(i) = \dim_k I^i/I^{i+1}$. This is called the Hilbert function of $A$ with respect to $I$. Since $\operatorname{gr}_I(A) \cong \oplus_{i \ge 0} I^i/ I^{i+1}$, where $I^0  = R$, it is not hard to see that $H_A(i) = \dim_{k} [\operatorname{gr}_I(A)]_i$, where $[ \; \;]_i$ denotes the $i$-th piece of a graded module. In other words, this is a natural way to pass a local object to a graded object.
If you are familiar with a blow-up, then it can be also seen as follows. Let $X = \operatorname{Spec}(A)$ and $Z = V(I)$. Let $X'$ the blowup of $X$ along $Z$, i.e., there exists a proper binational morphism $\pi: X' \to X$ such that it is an isomorphism outside $Z$. We call $E = \pi^{-1}(Z)$ the exceptional fiber of the blowup. Then $E \cong \operatorname{Proj} \operatorname{gr}_I(A)$. 
Another significance of the associated graded ring is that "good" properties transfer to the ring $A$. For instance, if $\operatorname{gr}_I(A)$ is reduced, domain, or integrally closed domain, so is $A$. Not only this, but also, regular, Gorensteinness, or Cohen-Macaulayness also transfer. I believe it was Hironaka who brought the significance of the associated graded algebra to study singularities. 
In commutative algebra, I believe, Craig Huneke is one of the first people who initiated the study. Using the following exact sequences
$$
0 \to IA[It] \to A[It] \to \operatorname{gr}_I(A) \to 0,
$$
and
$$
0 \to \oplus_{\ge 1} A[It] \to A[It] \to A \to 0,
$$
He showed that if $A$ and $A[It]$ are Cohen-Macaulay, then $\operatorname{gr}_I(A)$ is. Maybe it is worth putting at least one example how to use the Cohen-Macualayness of  $\operatorname{gr}_I(A)$. In studying the Hilbert functions, it is much simpler to study an Artinian graded ring over a field. In this case, Hilbert-function is nothing but the vector space dimension of each graded component. If $I = m$ and $\operatorname{gr}_I(A)$ is Cohen-Macaulay, then one can reduced to the good case, Artinian graded ring over a field. 
The name tangent cone might go back to Zariski since $m/m^2$ is called the Zariski cotangent space and its dual is the Zariski tangent space. The word cone probably comes from taking the affine cone of the projective scheme mentioned above. 
In your question, you analyzed the case that $I$ is a non zero principal and $A$ is an integral domain. However, the assumption $I$ is generated by a non zero divisor is enough. In fact, when $I$ is a complete intersection of height $s$, generated by a sequence of non zero divisors, then $\operatorname{gr}_I(A) \cong A/I[x_1, \dots, x_s]$ which generalizes the case when $I$ is principal. 
I think that last remark you made has to do with cotangent $m/m^2$ and conormal $I/I^2$ modules. Recently, there was an answer to this question in algebraic geometry in case $I$ is a complete intersection. However, I don't remember which post it was.
