Asymptotic behaviour of generalized binomial coefficient $\frac{an(an-1)...(an-n+1)}{n!}$ Let $a\in(0,1)$. What is the asymptotic behaviour of $\frac{an(an-1)...(an-n+1)}{n!}$ as $n\rightarrow\infty$?
It looks like it might be possible to express this in terms of gamma functions and use Stirling's approximation.
It is clear that $\frac{an(an-1)...(an-n+1)}{n!}=\frac{\Gamma(an+1)}{\Gamma(an-n+1)\Gamma(n+1)}$. But Stirling's approximation doesn't work for a gamma function with a negative argument.
 A: You can use the reflection formula and the known asymptotics for the gamma function to obtain
\begin{align*}
\frac{{\Gamma (an + 1)}}{{\Gamma (an - n + 1)\Gamma (n + 1)}} & = \frac{{\Gamma (an + 1)}}{{\Gamma (1 - (1 - a)n)\Gamma (n + 1)}} \\ & = \frac{{\sin (\pi (1 - a)n)}}{\pi }\frac{{\Gamma (an + 1)\Gamma ((1 - a)n)}}{{\Gamma (n + 1)}}\\ &
\sim \sqrt {\frac{{2a}}{{\pi (1 - a)n}}} \sin (\pi (1 - a)n)((1 - a)^{1 - a} a^a )^n ,
\end{align*}
as $n\to +\infty$ with any fixed $0<a<1$.
A: I'll try to present the raw idea. The quantity changes sign frequently. Let us take it by modulus. For simplicity, let us assume that $a$ is an irrational number. Thus, we have
$$
 S_n:=\left|\frac{an(an-1)...(an-n+1)}{n!}\right|=\frac{n^n}{n!}\prod_{0\leq\frac{k}{n}< a}\left(a-\frac{k}{n}\right)\cdot\prod_{a<\frac{k}{n}<1}\left(\frac{k}{n}-a\right).
$$
Taking the logarithm, we obtain
$$
 \ln S_n=n\ln n-\ln n!+\sum_{0\leq\frac{k}{n}< a}\ln\left(a-\frac{k}{n}\right)+\sum_{a<\frac{k}{n}<1}\ln\left(\frac{k}{n}-a\right).
$$
Now, we can use the Stirling approximation for $n!$, and the Riemann sum approximations for the integrals:
$$
 \ln S_n=n+n\int_0^a\ln x dx+n\int_0^{1-a}\ln x dx+o(n)=n(1+a\ln a-a+(1-a)\ln(1-a)-(1-a))+o(n)=n(a\ln a+(1-a)\ln(1-a))+o(n).
$$
We obtain the rough asymptotic. I think the main idea is more or less understandable - the technical details can be recovered without large troubles. More accurate asymptotics of the factorial, and, especially, the Riemann sums can give the next terms in the result. The sign of $S_n$ is something like $(-1)^{n-[an]}$.
