Proving that 11 cannot be written in the form $p^2+4pq+q^2$ I have to prove that it is imossible to write 11 in the form $p^2+4pq+q^2$ where $p$ and $q$ are integers. I tried to start a proof by contradiction like this:
$$
p^2+4pq+q^2 = 11\\
2(p+q)^2 - (p^2 + q^2) = 11\\
2(p+q)^2 = 11 + (p^2 + q^2)
$$
I'm really not sure where to go from there, though. A pointer in the right direction would be much appreciated!
 A: For variety, here's a method that doesn't require guessing at which small modulus to try. (however, it does use some more sophisticated mathematics)
First, we simplify our quadratic by completing the square
$$ (p+2q)^2 - 3q^2 = 11$$
Next, we homogenize:
$$  (pr+2qr)^2 = 11 r^2 + 3 (qr)^2$$
and for clarity, change variable
$$ z^2 = 11 x^2 + 3 y^2$$
(rational solutions to the original equation would be $q = y/x$ and $p = z/x - 2q$)
Now, we invoke a two relevant sledgehammers.
This equation obviously a real solution, and the only relevant primes to check are $3$ and $11$.
$$ (3, 11)_{3} = \left(\frac{11}{3}\right) = -1$$
$$ (3, 11)_{11} = \left(\frac{3}{11}\right) = 1$$
(and from this, we can also conclude $(3,11)_2 = -1$) where the left hand side is the Hilbert symbol and the right hand side is the Legendre symbol.
Since we got a $-1$, the equation has no rational solutions, and thus it has no integer solutions.
Incidentally, this result explains why the other posts were successful when checking modulo $3$ and checking modulo $8$.

A lowbrow summary of this method would be
Complete the square, find a real solution, then try working modulo 8,
then try working modulo every prime dividing one of the coefficients.
If you found solutions in all cases, then your quadratic has rational
solutions. Otherwise, it doesn't.

The main thing the sophisticated math tells you is that those are the only primes useful to check, and that if the check passes every test, then there truly are rational solutions. (actually proving none of the rational solutions are integer solutions is generally much trickier)
A: I'm a number theory noob, but taking all mod 4 keeping in mind what odd or even integers squares can yield mod 4, should solve it?! :)
A: Looking at $$2(p+q)^2 - (p^2 + q^2),$$ this will be even when $p$ and $q$ have the same parity.  But if $p$ and $q$ have different parity then modulo $4$ it is equivalent to $2\times 1 - 1 =1$, while $11 \equiv 3 \mod 4$. 
So $11$ cannot be expressed in that form.
A: In an introductory course in Number Theory, one of the early things one learns is that the sum of two squares cannot be congruent to $-1$ modulo $4$. This can be shown by a short enumeration of cases.
Modulo $4$, the middle term $4pq$ is irrelevant, and we are finished.    
We can also work modulo $3$, directly by a short consideration of cases, made even shorter by the symmetry.  
Or else we can reluctantly break symmetry and rewrite our expression as $(p+2q)^2-3q^2$, and note that modulo $3$ we need pay no attention to $3q^2$.  But the perfect square $(p+2q)^2$ is congruent to $0$ or $1$ modulo $3$, while $11$ is congruent to $-1$. 
Or else one can note that modulo $3$ we are looking at $p^2-2pq+q^2$, that is, $(p-q)^2$. 
Remark: The strategy we used is a common one.  When we are given a Diophantine equation, and suspect that it has no solutions, it is worthwhile to look at the equation modulo some small numbers. If for some $m$ the equation cannot hold modulo $m$, then it cannot hold in the integers. 
The simplest such arguments are parity arguments (working modulo $2$).  We can think of working modulo numbers greater than $2$ as generalized parity arguments.
