In general, what you claim is false. Imagine this example: let $\pi:\mathbb{A}^2=\text{Spec}k[x,y]\to \mathbb{A}^1=\text{Spec}k[x]$ be the projection on the first component. Fix a closed point $(a,b)$ in the affine plane (corresponding to the maximal ideal $(x-a,y-b)$), we have $\pi(a,b)=a$ in the affine line. Now, let's compare the local ring at $(a,b)$ with the one of the push forward: we have by definition that the local ring in the affine plane is
$$
\mathcal{O}_{A^2,(a,b)}=k[x,y]_{(x-a,y-b)}=\lim_{(a,b)\in U}\mathcal{O}_{A^2}(U)
$$
and, always applying the definition
$$
(\pi_*(\mathcal{O}))_a=\lim_{a\in V} \left( (\pi_*\mathcal{O}_{A^2})(V) \right)=\lim_{a\in V} \left( \mathcal{O}_{A^2}(\pi^{-1}(V) \right) \subseteq \lim_{\pi^{-1}(a)\subseteq U} \left( \mathcal{O}_{A^2}(U) \right)=\lim_{a\times \mathbb{A}^1\subseteq U} \left( \mathcal{O}_{A^2}(U) \right)
$$
In this second chain, I put a sign of subset because we don't even know that we are ranging over all possible open sets, but just over those which are preimages of opens of the line (in this specific example, however, we get an equality there, if we think about how the topologies are made). In any case, we only get
$$
\lim_{a\times \mathbb{A}^1\subseteq U} \left( \mathcal{O}_{A^2}(U)\right)=k[x,y]_{(x-a)} \subseteq\lim_{(a,b)\in U}\mathcal{O}_{A^2}(U)
$$
and the containment is strict in this case.
However, it is hard for me to think with certainty to conditions that make what you claim true (of course $\pi$ beign an isomorphism works, but I feel that it can be weakened). I feel that even if we require that the inverse image is just a point we don't have the guarantee that we range over enough open sets via the preimage (not all oopen neighbouroods of the point $p$ are of the form $\pi^{-1}(V)$, where $V$ is a neighbourood of $\pi(p)$).