Euler Product of Modular L-series

Question

This question is probably really easy to someone that knows the theory of modular forms well, so I apologize if this is obvious. Suppose $E_1$ and $E_2$ are elliptic curves over $\mathbb{Q}$ and the Euler product expansion of the Hasse-Weil L-series of each matches at all primes of good reduction. Some references use the definition of modular to be that for all but finitely many primes, $L(E_1, s)=L(f,s)$ where $f$ is a (certain type of) modular form. Some references seem to say that modular means $L(E_1, s)=L(f, s)$ where it seems implied that all factors are the same.

My first question is whether or not there is a way to see directly from modularity properties (such as there being an analytic continuation of $L(E_1, s)$ that satisfies a functional equation) that there is some unique way to fill in the rest of the Euler factors. The second related, or possibly equivalent, question is whether or not you can tell that $L(E_1, s)$ and $L(E_2, s)$ have the same factors just from knowing the factors of good reduction.

Note: the motivation is from more general "Galois representations coming from geometry" where you don't know something nice about the relation between the varieties. For instance, if you prove this by first saying they must be isogenous it won't generalize.

Note 2: the reason I find this hard is that a priori you could have bad reduction of $E_1$ and $E_2$ at exactly two primes $p_1$ and $p_2$, and somehow it just works out that the factor of $L(E_1, s)$ at $p_1$ is exactly the factor at $L(E_2, s)$ at $p_2$ and vice-versa to give exactly the same L-function, but the factors are filled in differently (this can't happen in the elliptic curve case because we know the formula explicitly, but again I'm wondering if it can be seen without knowledge of this formula).

If you don't like this wording of the question using elliptic curves and then not being able to use that fact, the real question is if $X$ is a variety over $\mathbb{Q}$ and its middle $\ell$-adic cohomology is $2$-dimensional with the property that the L-series coming from the (contragredient) Galois representation matches the L-series of a normalize Hecke eigenform for all primes of good reduction of $X$ (an integral model was fixed), does this uniquely determine the other factors in the sense that if I had another modular variety of the same dimension with the same modular form attached I could conclude every factor of the L-series matches with every factor of $L(X,s)$?

Answer 1

KCd's answer in the comments is superb, and I learned a lot from it. I spent a while thinking about how to do this from just representation theory without modular forms, and I wanted to record what I figured out.

Let $G = \mathrm{Gal}(\overline{\mathbb{Q}}/\mathbb{Q})$. Let $\rho: G \to V$ and $\sigma : G \to W$ be the $\ell$-adic $G$-representations on the two cohomology groups. For any prime $p$, let $I_p \subset D_p \subset G$ be an inertia group and decomposition group (they are defined up to conjugacy). So $D_p/I_p$ is generated by the Frobenius at $p$. As I am sure you know, for primes of good reduction other than $\ell$, the inertia group $I_p$ acts trivially on $V$ (respectively $W$) and the condition on the Euler factors means that $\mathrm{Frob}_p$ has the same characteristic polynomial on $V$ and on $W$. In other words, your hypothesis is that, for almost all $p$, it is true that for all $g \in D_p$ that $\rho(g)$ and $\sigma(g)$ have the same characteristic polynomial.

Claim: For all $g$ in $G$, the actions of $g$ on $V$ and $W$ have the same characteristic polynomial.

Proof: The coefficients of the characteristic polynomial are continuous functions $G \to \mathbb{Z}_{\ell}$, so it is enough to show that $\bigcup D_p$ is dense in $G$. A basis of open sets for $G$ is cosets $g N$, where $N$ is a finite index normal subgroup. For any such $g$ and $N$, let $K$ be the fixed field of $N$. The Cebatarov density theorem tells us that there are infinitely many primes for which $\mathrm{Frob}_p(K/\mathbb{Q})=g$. For these primes, $D_p \cap gN$ is nonempty. So $\bigcup D_p$ meets every open set, and is thus dense. $\square$

So, we have two representations that have the same characteristic polynomials, and we want to prove that they are isomorphic. This isn't always true. If $G$ were $\mathbb{Z}_{\ell}$, then we couldn't distinguish the trivial representation from $a \mapsto \left( \begin{smallmatrix} 1 & a \\ 0 & 1 \end{smallmatrix} \right)$ by characteristic polynomials. At this point, I pull out a theorem of Serre:
Theorem: If $E$ is a non-CM elliptic curve then, for $\ell$ sufficiently large, the map $G \to GL_2(\mathbb{Z}/\ell)$ is surjective.

I won't deal with the CM case; it should be easier. I will note that I can modify the argument to only use that $\mathbb{Q}_{\ell}[G] \to \mathrm{Mat}_{2 \times 2}(\mathbb{Q}_{\ell})$ is surjective, which I think is a lot easier to show.

Our situation is now the following: We have a group $G$ and two representations $\rho$ and $\sigma$ which we know to (a) have large images and (b) have the same characters. We want to know that $\rho$ and $\sigma$ are isomorphic.

Up to this point, everything was motivated. I now use a trick. Recall that the symmetric group $S_3$ is generated by $s$ and $t$ subject to $s^2=t^2=(st)^3=e$. The symmetric group embeds in $\mathrm{GL}(\mathbb{Z}_{\ell})$ with generators $\left( \begin{smallmatrix} 1 & 0 \\ -1 & -1 \end{smallmatrix} \right)$ and $\left( \begin{smallmatrix} 0 & 1 \\ 1 & 0 \end{smallmatrix} \right)$. Let $s$ and $t$ be preimages of these matrices under $\rho$.

Then $\sigma(s)$ has characteristic polynomial $\lambda^2-1$. Looking at possible Jordan forms, we see that $\sigma(s)^2=\mathrm{Id}$. Similarly, $\sigma(t)^2=\mathrm{Id}$ and $\sigma(st)^3 = \mathrm{Id}$. So $\sigma(s)$ and $\sigma(t)$ also generate a representation of $S_3$, with the same character as $\rho(s)$, $\rho(t)$. Since representations of finite groups are determined by their characters, there is an isomorphism $V \cong W$ making $\rho$ and $\sigma$ coincide on $\langle s, t \rangle$. We choose such an isomorphism for now on, and equate $S_3$ with its images in $GL(V) = GL(W)$.

For any $g$ in $G$, and $w \in S_3$, we have $\mathrm{Tr}(\rho(g) w) = \mathrm{Tr}(\sigma(g) w)$. (Lift $w$ to some $u \in \langle s,t \rangle$, and use the equality of characteristic polynomials for $gu$.) But the linear functions $X \mapsto \mathrm{Tr}(X w)$, for $w \in S_3$, span $\mathrm{Hom}(\mathrm{Mat}_{2 \times 2}, \mathbb{Q}_{\ell})$. So we deduce that $\sigma(g) = \tau(g)$, as desired.