Euler Product of Modular L-series

This question is probably really easy to someone that knows the theory of modular forms well, so I apologize if this is obvious. Suppose $E_1$ and $E_2$ are elliptic curves over $\mathbb{Q}$ and the Euler product expansion of the Hasse-Weil L-series of each matches at all primes of good reduction. Some references use the definition of modular to be that for all but finitely many primes, $L(E_1, s)=L(f,s)$ where $f$ is a (certain type of) modular form. Some references seem to say that modular means $L(E_1, s)=L(f, s)$ where it seems implied that all factors are the same.

My first question is whether or not there is a way to see directly from modularity properties (such as there being an analytic continuation of $L(E_1, s)$ that satisfies a functional equation) that there is some unique way to fill in the rest of the Euler factors. The second related, or possibly equivalent, question is whether or not you can tell that $L(E_1, s)$ and $L(E_2, s)$ have the same factors just from knowing the factors of good reduction.

Note: the motivation is from more general "Galois representations coming from geometry" where you don't know something nice about the relation between the varieties. For instance, if you prove this by first saying they must be isogenous it won't generalize.

Note 2: the reason I find this hard is that a priori you could have bad reduction of $E_1$ and $E_2$ at exactly two primes $p_1$ and $p_2$, and somehow it just works out that the factor of $L(E_1, s)$ at $p_1$ is exactly the factor at $L(E_2, s)$ at $p_2$ and vice-versa to give exactly the same L-function, but the factors are filled in differently (this can't happen in the elliptic curve case because we know the formula explicitly, but again I'm wondering if it can be seen without knowledge of this formula).

If you don't like this wording of the question using elliptic curves and then not being able to use that fact, the real question is if $X$ is a variety over $\mathbb{Q}$ and its middle $\ell$-adic cohomology is $2$-dimensional with the property that the L-series coming from the (contragredient) Galois representation matches the L-series of a normalize Hecke eigenform for all primes of good reduction of $X$ (an integral model was fixed), does this uniquely determine the other factors in the sense that if I had another modular variety of the same dimension with the same modular form attached I could conclude every factor of the L-series matches with every factor of $L(X,s)$?

-
First of all, the local factor at $p$ is a reciprocal polynomial in $p^{-s}$, so unless the local factors at two different primes are both 1, they are definitely not equal (special feature of Euler product over rational primes). As for the uniqueness of being able to fill in a set of missing factors, the analytic continuation and functional equation explain this. The basic idea is this: if $F(s)$ is a hypothetical completed $L$-function, then it is entire and $F(2-s) = wF(s)$, where $w = \pm 1$. Changing a finite number of Euler factors will turn this into $G(s) = R(s)F(s)$, where [contd.] –  KCd Mar 3 '12 at 1:24
$R(s) = \prod (1-\alpha_i p_i^{-s})/\prod (1 - \beta_j q_j^{-s})$ is a ratio of finite products of terms of the form $1-cp^{-s}$ for primes $p$ and nonzero complex numbers $c$. (Any reciprocal polynomial in $p^{-s}$ with constant term 1 can be factors into such parts.) Some $p_i$'s or $q_j$'s may be equal. Now suppose $G(s)$ is entire and $G(2-s) = uG(s)$, where $u = \pm 1$. Then $R(s) = G(s)/F(s)$ satisfies $R(2-s) = \varepsilon R(s)$ where $\varepsilon = \pm 1$. Show by induction on the number of factors in $R(s)$ that $R(s)$ is constant. –  KCd Mar 3 '12 at 1:28
Note, to avoid a misunderstanding, that all I'm saying above is that there is an elementary argument showing there's at most one way to fill in Euler factors to get an entire function with a particular functional equation. By no means does the argument I just sketched tell you what a correct choice of missing Euler factors is supposed to be. Also, maybe I'm forgetting something and other known features of the completed $L$-function should be used in a solution, but essentially it boils down to seeing that nonconstant ratios like $R(s)$ don't satisfy expected functional equations. –  KCd Mar 3 '12 at 1:30
@KCd Thanks a lot. This is really helpful. –  Matt Mar 5 '12 at 17:28

KCd's answer in the comments is superb, and I learned a lot from it. I spent a while thinking about how to do this from just representation theory without modular forms, and I wanted to record what I figured out.

Let $G = \mathrm{Gal}(\overline{\mathbb{Q}}/\mathbb{Q})$. Let $\rho: G \to V$ and $\sigma : G \to W$ be the $\ell$-adic $G$-representations on the two cohomology groups. For any prime $p$, let $I_p \subset D_p \subset G$ be an inertia group and decomposition group (they are defined up to conjugacy). So $D_p/I_p$ is generated by the Frobenius at $p$. As I am sure you know, for primes of good reduction other than $\ell$, the inertia group $I_p$ acts trivially on $V$ (respectively $W$) and the condition on the Euler factors means that $\mathrm{Frob}_p$ has the same characteristic polynomial on $V$ and on $W$. In other words, your hypothesis is that, for almost all $p$, it is true that for all $g \in D_p$ that $\rho(g)$ and $\sigma(g)$ have the same characteristic polynomial.

Claim: For all $g$ in $G$, the actions of $g$ on $V$ and $W$ have the same characteristic polynomial.

Proof: The coefficients of the characteristic polynomial are continuous functions $G \to \mathbb{Z}_{\ell}$, so it is enough to show that $\bigcup D_p$ is dense in $G$. A basis of open sets for $G$ is cosets $g N$, where $N$ is a finite index normal subgroup. For any such $g$ and $N$, let $K$ be the fixed field of $N$. The Cebatarov density theorem tells us that there are infinitely many primes for which $\mathrm{Frob}_p(K/\mathbb{Q})=g$. For these primes, $D_p \cap gN$ is nonempty. So $\bigcup D_p$ meets every open set, and is thus dense. $\square$

So, we have two representations that have the same characteristic polynomials, and we want to prove that they are isomorphic. This isn't always true. If $G$ were $\mathbb{Z}_{\ell}$, then we couldn't distinguish the trivial representation from $a \mapsto \left( \begin{smallmatrix} 1 & a \\ 0 & 1 \end{smallmatrix} \right)$ by characteristic polynomials. At this point, I pull out a theorem of Serre:
Theorem: If $E$ is a non-CM elliptic curve then, for $\ell$ sufficiently large, the map $G \to GL_2(\mathbb{Z}/\ell)$ is surjective.

I won't deal with the CM case; it should be easier. I will note that I can modify the argument to only use that $\mathbb{Q}_{\ell}[G] \to \mathrm{Mat}_{2 \times 2}(\mathbb{Q}_{\ell})$ is surjective, which I think is a lot easier to show.

Our situation is now the following: We have a group $G$ and two representations $\rho$ and $\sigma$ which we know to (a) have large images and (b) have the same characters. We want to know that $\rho$ and $\sigma$ are isomorphic.

Up to this point, everything was motivated. I now use a trick. Recall that the symmetric group $S_3$ is generated by $s$ and $t$ subject to $s^2=t^2=(st)^3=e$. The symmetric group embeds in $\mathrm{GL}(\mathbb{Z}_{\ell})$ with generators $\left( \begin{smallmatrix} 1 & 0 \\ -1 & -1 \end{smallmatrix} \right)$ and $\left( \begin{smallmatrix} 0 & 1 \\ 1 & 0 \end{smallmatrix} \right)$. Let $s$ and $t$ be preimages of these matrices under $\rho$.

Then $\sigma(s)$ has characteristic polynomial $\lambda^2-1$. Looking at possible Jordan forms, we see that $\sigma(s)^2=\mathrm{Id}$. Similarly, $\sigma(t)^2=\mathrm{Id}$ and $\sigma(st)^3 = \mathrm{Id}$. So $\sigma(s)$ and $\sigma(t)$ also generate a representation of $S_3$, with the same character as $\rho(s)$, $\rho(t)$. Since representations of finite groups are determined by their characters, there is an isomorphism $V \cong W$ making $\rho$ and $\sigma$ coincide on $\langle s, t \rangle$. We choose such an isomorphism for now on, and equate $S_3$ with its images in $GL(V) = GL(W)$.

For any $g$ in $G$, and $w \in S_3$, we have $\mathrm{Tr}(\rho(g) w) = \mathrm{Tr}(\sigma(g) w)$. (Lift $w$ to some $u \in \langle s,t \rangle$, and use the equality of characteristic polynomials for $gu$.) But the linear functions $X \mapsto \mathrm{Tr}(X w)$, for $w \in S_3$, span $\mathrm{Hom}(\mathrm{Mat}_{2 \times 2}, \mathbb{Q}_{\ell})$. So we deduce that $\sigma(g) = \tau(g)$, as desired.

-