A probability measure defined on a sample space $\Omega$ has the following properties:

1. For each $E \subset \Omega$, $0 \le P(E) \le 1$
2. $P(\Omega) = 1$
3. If $E_1$ and $E_2$ are disjoint subsets $P(E_1 \cup E_2) = P(E_1) + P(E_2)$

The above definition defines a measure that is finitely additive (by induction) but not necessarily countably additive.

What is a probability measure that would be finitely additive but not countably additive (for a countable sample space $\Omega$)?

The example that I have seen most commonly on forums (this and elsewhere) is to set $P(E) = 0$ if $E$ is finite and $P(E) = 1$ if $E$ is co-finite. But that is not a probability measure as defined above since it is not defined on every subset of $\Omega$.

So an example of such a probability measure, or what is the reasoning that a finitely additive probability measure is not always countably additive?

-
I doubt that a general (probability-) measure has to be defined on all subsets of $\Omega$. If it is defined for a set $A$, then $A$ is said to be measurable. –  Berci Sep 30 '12 at 13:41
@Berci I believe that would make sense for uncountable sets and the restriction exists because a general measure is countably additivity. But for a countable set, you should be able to define a probability measure for each subset. I edited the question to make it clarify countable sample space. –  Kostub Deshmukh Oct 1 '12 at 6:31

Let $\mathcal{U}$ be a free ultrafilter on $\mathbb{N}$. Let $P(A)=1$ if $A\in\mathcal{U}$ and $P(A)=0$ if $A\notin\mathcal{U}$. I think it is impossible to give an explicit example of a finitely additive measure on a $\sigma$-algebra that is not countably additive, but our resident set theorists might be able to tell you more about that.

-
Thanks very helpful. I can see why a free ultrafilter will have this property, while a principle ultrafilter will not. Moreover, the Ultrafilter Lemma asserts the existence of a free ultrafilter (which contains the co-finite filter). However I cannot quite visualize what such a free ultrafilter on $\N$ will contain. And why can't such an ultrafilter be explicitly described? –  Kostub Deshmukh Oct 1 '12 at 6:04
@kostub: There are models without the axiom of choice in which there are no free ultrafilters (at all!), so it is impossible to give an explicit description of a free ultrafilter. Using the axiom of choice we can prove such filter exists, but not much more. –  Asaf Karagila Oct 1 '12 at 7:33

In the following note the author shows that a finitely additive diffused measure on $\mathcal{P}(\omega)$ can be used to define a non Ramsey family. Combining this with a result of Mathias, it follows that it is consistent with $ZFC$ that there is no (ordinal) definable finitely additive diffused total measure on $\mathcal{P}(\omega)$.

-

In this interesting note, the author proves the following: It is consistent to have a finitely additive total extension of Lebesgue measure on $[0, 1]$ such that, although, the measure zero sets form a sigma ideal, there is no real valued measurable cardinal below continnum.

-