Probability that $x \equiv 3 \pmod{4}$ I am working on a number theory project and I am interested in the following statement:

What is the probability that an integer $x$ has the property that $|x| \equiv 3 \mod{4}$? 

This seems equivalent to me as asking, what is the probability that the Legendre symbol $\left(\frac{-1}{p}\right) = -1$, by Quadratic Reciprocity.
Additionally, since we know from the integers that the probability a number in $\mathbb{Z}$ is divisible by a prime $p$ is $1/p$, can we say that the probability that a Gaussian integer, $a + bi \in \mathbb{Z}[i]$ with $a,b \neq 0$, is divisible by a Gaussian prime is $1/(a^2 + b^2)$? (Notice the theorem regarding Gaussian primes here: Gaussian Primes).
These may seem like disjoint questions, but they are related to the fact that the probability that two gaussian integers are coprime involves the zeta function.
 A: From my comment above, now relocated here:
There are four equivalence classes modulo $4$. Every integer falls into one of those equivalence classes. The equivalence class $[3] = \{n: n = 4k + 3, k\in \mathbb Z\}$. Every fourth integer belongs to $[3]$; that is, consecutive integers map cyclicly into one of four equivalence classes: $$4k \mapsto [0], \;\; 4k+1 \mapsto [1], \;\; 4k + 2 \mapsto [2], \;\; 4k + 3 \mapsto [3],\;\; 4k+ 4 = 4(k+1) \mapsto [0], \cdots$$
Hence the probability that an integer is equivalent to $3 \pmod 4$ is no more than probability of that integer belonging to one of four equivalence classes: probability $\;\large \frac 14$. 
I am not at all clear as to how this relates to your second question.
A: The probability that a rational integer (i.e. in $\bf Z$) has residue $r$ modulo $n$ is $1/n$, since the probability attached to each residue is equal and there are $n$ residues. Here we are using an asymptotic notion of "probability distribution" as mentioned in the comments.
Similarly, in any number field $K/{\bf Q}$ with ring of integers ${\cal O}_K$ and ideal ${\frak n}\triangleleft {\cal O}_K$, any intuitively nice asymptotic notion of density would be translation-invariant, so the probability an integer $\in{\cal O}_K$ would be in a fixed residue $r$ mod $\frak n$ (i.e. congruent to $r+{\frak n}$ in the quotient ${\cal O}_K/{\frak n}$) is the same for every choice of $r$, and hence is equal to the reciprocal of the number of equivalence classes, which is the order $|{\cal O}_K/{\frak n}|$ of the quotient, which is (by definition) the norm $N({\frak n})$ of $\frak n$. If ${\frak n}=(n)$ is a principal ideal generated by some integer $n\in{\cal O}_K$, then $N({\frak n})=N_{K/{\bf Q}}(n)$ is the norm of $n$ (this is a fact from algebraic number theory). This is indeed $a^2+b^2$ for $n=a+bi\in\href{http://en.wikipedia.org/wiki/Gaussian_integer}{{\bf Z}[i]}={\cal O}_{{\bf Q}(i)}$.
The same heuristic for deducing that the probability two rational integers are coprime works for computing the probability two integers $\in{\cal O}_K$ of a number field $K$ are coprime (with the caveat that $\cal O$ is a UFD). Originally, we assume that the events "being divisible by the prime $p$" for distinct $p$ are independent, and two numbers are coprime iff for each prime $p$ it is not the case that both are divisible by $p$ (which has probability $1/p^2$), so the probability two rational integers are coprime is given by the Euler product factorization associated with the Riemann zeta function:
$$\prod_p\left(1-\frac{1}{p^2}\right)=\frac{1}{\zeta(2)}=\frac{6}{\pi^2}.$$
Similarly, the probability two integers in ${\cal O}_K$ are coprime is
$$\prod_{\frak p}\left(1-\frac{1}{N({\frak p})^2}\right)=\frac{1}{\zeta_K(2)}.$$
See Dedekind zeta function.
On Mathworld, for Gaussian and Eisenstein integers the probabilities are listed in closed form.
