Filling in a Gap in the Proof of the Converse to Modularity Here is the theorem:
Let $f = \sum_{n = 1}^\infty a_nq^n \in S_2(\Gamma_0(N))$ be a newform with rational coefficients. Then there is a (unique up to $\mathbb Q$-isogeny) elliptic curve $E/\mathbb Q$ of conductor $N$ such that $f_E = f$, where $f_E := \sum_{n = 1}^\infty a_n(E)q^n$.
Milne's Elliptic Curves, page 222, gives the most detailed proof of this converse to the Modularity Theorem that I have seen. I have been able to fill in all of the details, except the general case. Namely, Milne specifies to the case where $E_f$, the canonical abelian variety attached to $f$ via the Shimura construction (which in this case is an elliptic curve over $\mathbb Q$) is equal to $X_0(N)$. Then he gives a proof of the theorem using Eichler-Shimura, which I understand.
However, he ends with the following "proof" for the general case: "The proof of the general case is very similar except that, at various places in the argument, an elliptic curve has to be replace either by a curve or the Jacobian variety of a curve." No references or helpful additional details about what these mysterious curves are are given. Can someone either point me to a complete proof, or proof the general case from the special case here?
 A: Here’s the argument I know. More precisely, I show that $f$ and the elliptic curve have the same $L$-function up to finitely many factors. If one accepts that the modular Jacobian $J_0(N)$ has good reduction outside $N$, these factors are only the primes dividing $N$. I don’t know how to deal with this final case – maybe Milne explains it – although I don’t think it’s the most significant part.
Let $T$ be the Hecke algebra over $\mathbb{Z}$ acting on modular forms for $\Gamma_0(N)$. We have a prime ideal $I_f \subset T$ such that $T_n-a_n(f) \in I$ for all $n \geq 1$ and $T/I_f \cong \mathbb{Z}$.
Define $A_f=J_0(N)/I_fJ_0(N)$. By the discussion of Diamond and Shurman in Section 6.6, $A_f$ is a rational elliptic curve.
Moreover, if $p$ does not divide $N$ and is of good reduction for $A_f$ (the second condition is redundant, but excludes only a finite number of primes anyway), then $T_p$ acts on $A_f$ mod $p$ by $Frob_p+Ver_p$ by Eichler-Shimura.
In particular, if $\ell \neq p$ is any prime, as the reduction mod $p$ is an isomorphism for the $\ell^n$-torsion points, it follows that $T_p$ acts on $A_f[\ell^n]$ by $Frob_p+Ver_p$. Since $A_f$ is an elliptic curve, we know that on $T_{\ell}A_f$, $Frob_p+Ver_p$ acts by $a_p(A_f)$.
But by definition, $T_p$ acts on $A_f$ by $a_p(f)$, so that $a_p(f)=a_p(A_f)$, QED.
