TL;DR: on any distinguished open $D(f)$ of $X$, both sheaves have sections isomorphic to the localization $(A/\mathfrak{a})_{\overline{f}}$. The rest of this answer explains this computation, and why the isomorphisms commute with the restriction maps on each sheaf.
This mainly boils down to understanding what the sections of each sheaf are over a distinguished base for the topology on $X$, which (in your case) should amount to using a few key propositions proved in Hartshorne. Let $M = A/\mathfrak{a}$, and let $\tilde{M}$ be the $\mathcal{O}_{X}$-module associated to the $A$-module $M$. We want to show that there is an isomorphism of $\mathcal{O}_{X}$-modules $i_{\ast}\mathcal{O}_{Y} \to \tilde{M}$; to do so, it suffices to build an isomorphism of $\mathcal{O}_{X}$-modules on a basis for the topology on $X$. (If you are not familiar with the technique of working with sheaves on a base, it may be helpful to consult section 2.7 of Vakil's wonderful algebraic geometry notes.)
Since $X$ is an affine scheme, there is a natural choice for the a basis for the Zariski topology on $X$, namely the distinguished opens of $X$. Let's compute the sections over a distinguished open of $X$ for each sheaf; we will see that they are the same. Suppose $D(f)$ is a distinguished open of $X$ for some $f \in A$. Depending on your definition of $\tilde{M}$, it is either a definition $\tilde{M}(D(f)) = M_{f}$, or a basic proposition that there is an isomorphism $\alpha_{f} \colon \tilde{M}(D(f)) \cong M_{f}$ of $\mathcal{O}_{X}(D(f))$-modules. I suspect you are in the latter setting since you are using Hartshorne, in which case I refer you to Proposition II.5.1(c). Since the $A$-module structure on $M$ is induced by the canonical quotient map $A \to A/\mathfrak{a}$, we see that $\tilde{M}(D(f)) \cong M_{f} = (A/\mathfrak{a})_{\overline{f}}$, where $\overline{f}$ denotes the class of $f$ in $A/\mathfrak{a}$.
On the other hand, let $i \colon Y \to X$ be the morphism of affine schemes induced by the canonical quotient morphism $i^{\sharp} \colon A \to A/\mathfrak{a}$. Then $i^{-1}(D(f)) = D(i^{\sharp}(f))$, so $i_{\ast}\mathcal{O}_{Y}(D(f)) = \mathcal{O}_{Y}(D(i^{\sharp}(f))) \cong (A/\mathfrak{a})_{\overline{f}}$, where the last isomorphism follows from Hartshorne Proposition II.2.2(c); I will call this last isomorphism $\beta_{f}$.
Hence, in each case, we have $i_{\ast}\mathcal{O}_{Y}(D(f)) \cong (A/
\mathfrak{a})_{\overline{f}} \cong \tilde{M}(D(f))$ as $\mathcal{O}_{X}(D(f))$-modules. Furthermore, this isomorphism commutes with restriction for any basic open; indeed, if $D(g) \subset D(f)$ for some $g, f \in A$, then $(A/
\mathfrak{a})_{\overline{g}}$ is a localization of $(A/
\mathfrak{a})_{\overline{f}}$, and the restriction maps $i_{\ast}\mathcal{O}_{Y}(D(f)) \to i_{\ast}\mathcal{O}_{Y}(D(g)), \tilde{M}(D(f)) \to \tilde{M}(D(g))$ commute with the canonical localization map $\rho \colon (A/
\mathfrak{a})_{\overline{f}} \to (A/
\mathfrak{a})_{\overline{g}}$. That is, we have a commutative diagram:
$$\require{AMScd}
\begin{CD}
i_{\ast}\mathcal{O}_{Y}(D(f)) @>\alpha_{f}>> (A/
\mathfrak{a})_{\overline{f}} @>\beta_{f}^{-1} >> \tilde{M}(D(f)) \\
@V~VV @V\rho VV @V~VV \\
i_{\ast}\mathcal{O}_{Y}(D(g)) @>\alpha_{g}>> (A/
\mathfrak{a})_{\overline{g}} @>\beta_{g}^{-1} >> \tilde{M}(D(g)) \\
\end{CD}
$$
where the left and right-most vertical maps are restriction maps. (This should be evident from the proofs of the aforementioned propositions in Hartshorne.) Hence, we have defined an isomorphism $i_{\ast}\mathcal{O}_{Y} \to \tilde{M}$ of $\mathcal{O}_{X}$-modules on a base, which is given over each distinguished open $D_{f}$ by $\beta_{f}^{-1} \circ \alpha_{f}$. This extends uniquely to an isomorphism $i_{\ast}\mathcal{O}_{Y} \to \tilde{M}$ of $\mathcal{O}_{X}$-modules, as desired.
