Conditional expectation of a Geometric random variable conditioned on the event $A=\{X>3\}$

Let $X$ be a geometric random variable and let $A$ denote the event $\{X>3\}$ find the conditional probability mass function of $X$ with respect to the event $A$ and then compute $E[X\mid A]$.

-

We have that $$P(X=k)=p(1-p)^k,\quad k=0,1,2,\ldots$$ and hence $P(X>3)=(1-p)^4$. Now if $k=0,1,2,3$, then of course $P(X=k\mid X>3)=0$ and if $k=4,5,\ldots$, then $$P(X=k\mid X>3)=\frac{P(X=k,X>3)}{P(X>3)}=\frac{P(X=k)}{P(X>3)}=p(1-p)^{k-4}.$$ The conditional expectation is thus given by $$E[X\mid X>3]=\sum_{k=4}^\infty kP(X=k\mid X>3)=\sum_{k=4}^\infty kp(1-p)^{k-4}=\sum_{k=0}^\infty (k+4)p(1-p)^k=4+E[X].$$
There is something wrong here. Conditioned on $X>3$, the conditional expectation of $X$ is $3+E[X]$, not $E[X]$. If $p = 0.5$ so that $E[X]=2$, why should the average value of $X$ be $2$ when we are given that $X > 3$? – Dilip Sarwate Dec 14 '12 at 3:33
I edited my answer. Though I'm getting $4+E[X]$, but I think it depends on whether you define the geometric distribution on $\{0,1,2,\ldots\}$ or on $\{1,2,\ldots\}$. – Stefan Hansen Dec 14 '12 at 3:45