# Orthogonality of the Gegenbauer Polynomials

Typically the orthogonality relation for the Gegenbauer polynomials is given as:

$$\int_{-1}^{1}C_{n}^{\alpha}(x)C_{m}^{\alpha}(x)\cdot(1-x^2)^{\alpha-1/2}dx=\frac{\pi2^{1-2\alpha}\Gamma(2\alpha+n)}{n!(n+\alpha)(\Gamma(\alpha))^2}\delta_{mn}$$

The Rodrigues formula for $C_{n}^{\alpha}(x)$ is given by:

$$C_{n}^{\alpha}(x)=constant\cdot(1-x^2)^{-\alpha+1/2}\frac{d^n}{dx^n}\left[(1-x^2)^{n+\alpha-1/2}\right]$$

My question is the following:
How do I derive/proove the orthogonality relation?

In George Andrews' "Special Functions" Ch. 6, he proves the orthogonality relation for the Hermite and Laguerre polynomials by substituting the Rodrigues form of the polynomial in for one of the polynomials and then using integration by parts $n$ times. However when I try that I get something like:

$$\sum_{k=1}^{m}(-1)^{k+1}C_{n-(k-1)}^{\alpha+(k-1)}(x)\frac{d^{m-k}}{dx^{m-k}}\left[(1-x^2)^{m+\alpha-1/2}\right]\bigg|_{-1}^{1}+\int_{-1}^{1}C_{n-m}^{\alpha+m}(x)\cdot(1-x^2)^{m+\alpha-1/2}dx$$

I don't immediately see why all the terms in the summation go to zero, nor how to evaluate the integral directly. Any ideas?

• I noticed that if you substitute the Rodrigues formula in for $C_{n-m}^{\alpha+m}(x)$ in the integral of my result (equation 3) it evaluates to $constant\cdot\frac{d^{(n-m-1)}}{dx^{9n-m-1)}}\left[(1-x^2)^{n+\alpha-1/2}\right]\bigg|_{-1}^{1}$, but I still don't know how to evaluate this though. – okj Jan 12 '12 at 17:45

Ok, I figured it out. You first expand $(1-x^2)^{m+\alpha-1/2}=\sum_{k=0}^{\infty}(-1)^k{m+\alpha-1/2\choose k}x^{2k}$, using the binomial theorem. You can then take the $(m-k)$-th derivative explicitly, which is $x^{-(m-k)}\sum_{k=0}^{\infty}(-1)^k{n+l-1/2 \choose k}(2k)_{m-k}x^{2k}$, at $x=\pm1$ this summation converges to zero unless $k=m$, so only the last term in the sum is left and it also goes to zero since both $(1-1^2)^{m+\alpha-1/2}$ and $(1-(-1)^2)^{m+\alpha-1/2}$. The integral can then be evaluated by invoking Rodrigues' formula again.