# Integral evaluation to get the characteristic function of the Gamma distribution

For parameters $$k\in \mathbb{N}$$ and $$\lambda\in\mathbb{R}$$, the density function of the gamma dist is $$f_{(k,\lambda)}(x)=\frac{x^{k-1}\lambda^ke^{-\lambda x}}{(k-1)!}$$

Then the characteristic function is

\begin{align*} \Phi_X(\omega)&=\int_0^\infty e^{i\omega x}f_{(k,\lambda)}(x)dx\\ &=\frac{\lambda^k}{(k-1)!}\int_0^\infty x^{k-1}e^{(i\omega-\lambda)x}dx \end{align*}

Here I applied an identity for the integral above, given as a hint in the question: $$\frac{\lambda-i\omega}{k}\int_0^\infty x^{k}e^{(i\omega-\lambda)x}dx$$

Then I made a change of variable $$(i\omega-\lambda)x=-y$$, which changed the integration limits from $$0$$ and $$\infty$$, to $$0$$ and $$-\infty$$ respectively. This is where I have some doubts if it's correct. So the characteristic function came to be $$\frac{\lambda^k}{k!(\lambda-i\omega)^k}\int_0^{-\infty}y^ke^{-y}dy$$

Now if the upper limit of the integral was $$+\infty$$, then the integral would be $$\Gamma(k+1)=k!$$ and I would've gotten the desired result. Where is it going wrong?

First of all, $$\lambda$$ must be positive for everything to make sense.
Next, as you suspect, when computing $$\int_0^\infty x^{k}e^{(i\omega-\lambda)x}\,dx$$, after the substitution $$(i\omega-\lambda)x=-y$$, the "limits" are not $$0$$ and $$-\infty$$; rather, the integration path is the ray $$\{re^{i\phi} : r\geqslant 0\}$$, where $$\phi=\arg(\lambda-i\omega)$$. To see that such an integral is equal to $$\int_0^\infty$$ of the same integrand, apply CIT to the sector $$\{re^{i\theta} : r\in[0,R],\theta\in[0,\phi]\};$$ as $$R\to+\infty$$, the integral along the circular arc part vanishes.
Alternatively, one may just note that $$z\mapsto\int_0^\infty x^k e^{-zx}\,dx$$ is analytic in $$\Re z>0$$, so that, since it is equal to $$k!/z^{k+1}$$ when $$z$$ is real (positive), the same is true for any $$z$$ (with $$\Re z>0$$) by analytic continuation.