Formally smooth morphisms lift against reduction Let $A\to B$ be a formally smooth of rings, that is for every $A$-algebra $T$ and every square zero ideal $I\subset T$ we have that $A\text{-alg}(B,T)\to A\text{-alg}(B,T/I)$ is surjective.
Now I seem to recall often reading that $A\to B$ being formally smooth should already imply that $A\text{-alg}(B,T)\to A\text{-alg}(B,T_\text{red})$ is surjective, where $T_\text{red}$ is the quotient of $T$ by the nilradical.
If $A$ is noetherian and $A\to B$ is even smooth, so in particular of finite type, I think i have a proof, but is it also true in general? Maybe if we restrict $T$ to be of finite type of of finite presentation over $A$?
Also, as a bonus, since I really wanted to ask this in the context of $C^\infty$-rings, I would like to know how much one can prove without using Hilberts basis theorem.
 A: This is false without some finiteness assumption. Indeed any surjection $A \to B$ with $I^2 = I$ where $I = \mathrm{ker}(A \to B)$ is formally smooth. Note that this can never happen if $I$ is finitely generated but can happen for infinitely generated $I$.
To produce a counterexample it suffices to find a ring $A$ with idempotent nilradical such that $A \to A_{red}$ does not split. Then $A \to A_{red}$ is formally smooth but there is no lift of the identity $A_{red} \to A_{red}$ along $A \to A_{red}$. Note in particular in this case
I think an example is given by $A = \mathbb{Z}_p[\{p^{\frac{1}{n}}\}_n]/(p^2)$. Any power of $p$ is nilpotent since $(p^{a/b})^{2b} = (p^2)^a = 0$ in $A$ so the nilradical $N = (\{p^{r}\}_{r \in \mathbb{Q}_{>0}})$ and $A/N = \mathbb{F}_p$. Moreover, $p^r = (p^{r/2})^2$ for any $r \in \mathbb{Q}_{>0}$ so $N^2 = N$ but $p \neq 0$ in $A$ so there is no splitting of $A \to A/N$. In this example $T = A$ is finitely presented as an $A$ algebra and $A \to B = A_{red}$ is of finite type but it still fails because $A \to B$ is not finitely presented.
As Cranium Clamp mentioned in the comments, it is ok if $T$ is Noetherian or more generally if the nilradical $N$ is nilpotent (which is the case whenever $N$ is finitely generated). In this case, if say $N^m = 0$, then $T/N^{k+1} \to T/N^k$ is square zero and so we can inductively lift along this extension, starting with $T/N^2 \to T/N = T_{red}$ to get a lift to $T/N^m = T$.
Another case that I think should be ok is if $A \to B$ is smooth not just formally smooth. Then by Noetherian approximation there exists a Noetherian ring $A_0$, a map $A_0 \to A$, and a finitely presented smooth map $A_0 \to B_0$ such that $B = A \otimes_{A_0} B_0$. Then it suffices to construct a lift of $B_0 \to T$ so without loss of generality we may assume that $A$ is Noetherian. I wrote the next paragraph before I noticed that OP has a proof when $A$ is Noetherian and $A \to B$ is finitely presented but I'll leave it for the benefit of others.
Next we can write $T = \mathrm{colim} T_\lambda$ where $T_\lambda$ are finitely presented $A$-algebras and the colimit is filtered. Letting $N$ denote the nilradical of $T$ and $N_{\lambda}$ the nilradical of $T_\lambda$, we have that $N = \mathrm{colim} N_\lambda$ (since every element of $T$ is contained in $T_\lambda$ for some $\lambda$ and its nilpotent if and only if it is nilpotent as an element of $T_{\lambda}$) and $T_{red} = T/N = \mathrm{colim}T_\lambda/N_{\lambda}$ (since filtered colimit is exact). Now since $B$ is a finitely presented $A$-algebra, we have
$$
\mathrm{colim}_\lambda Hom_{Alg_A}(B,R_\lambda) \to Hom_{Alg_A}(B,R)
$$
for any filtered colimit of $A$-algebras $R = \mathrm{colim} R_\lambda$. Thus our map $B \to T_{red}$ factors through $B \to T_\lambda/N_\lambda$ for some $\lambda$ which we can lift to $T_{\lambda}$ by the previous case since $T_{\lambda}$ is Noetherian.
Another case I think works is if $A \to B$ is a formally smooth map with $A$ and $B$ both Noetherian. Then by a deep theorem of Popescu, $B$ is a filtered colimit of smooth $A$-algebras $B_{\lambda}$ so we know we can lift $B_{\lambda} \to T_{red}$ to $T$ for each $\lambda$. Some care needs to be taken to pick compatible lifts for different $\lambda$ but I think this should be doable.
An interesting case is if $A$ is Noetherian and $A \to B$ is formally smooth but not smooth and $B$ is not Noetherian. There are such examples where $B$ is even flat over $A$ but is not the filtered colimit of smooth $A$-algebras (see this question and answer). I would guess that such examples might also lead to counterexamples to lifting along $T \to T_{red}$ but I'm not sure.
