Integral of $x^\alpha$ for Various $\alpha$ How can I prove this? I know why it's the case, but I can't prove it.
Give all $\alpha \in \mathbb R$ such that:
a)
$\displaystyle\int_a^b x^\alpha dx$ is convergent
b)
$\displaystyle\int_1^\infty x^\alpha dx$ is convergent
 A: I will do part b) in detail:
Here, you have an improper integral, since the interval of integration has infinite length.
By definition, the integral $\int_1^\infty x^\alpha\, dx$ is convergent if and only if 
$$
\tag{0}\lim_{b\rightarrow \infty}\int_1^b x^\alpha\, dx
$$
exists (in the finite sense); in which case we say the integral converges to the value of the limit. Note that you evaluate the integral above first; this will give you an expression in $b$. Then you take the limit or demonstrate that it does not exist.
The picture below, which shows the scenario for $\alpha<-1$ and in which the gray shaded region represents $\int_1^b x^\alpha\,dx$,  may help one to see why we define $\int_1^\infty x^\alpha\,dx$ in 
this way:


Now on to computing the required limit. We need to determine when the limit in $(0)$ exists.
Of course, this will likely depend on the particular value of $\alpha$.
Let's deal with a special case first: $\alpha=-1$. 
In this case, we have
$$
\lim_{b\rightarrow\infty}\int_1^b x^{-1}\, dx
=\lim_{b\rightarrow\infty} \ln |x|\bigl|_1^b =\lim_{b\rightarrow\infty} \ln b=\infty.
$$ 
Thus, for $\alpha=-1$, the improper integral $\int_1^\infty x^\alpha\,dx$ does not converge.
If $\alpha\ne -1$, then:
$$\tag{1}\eqalign{
\lim_{b\rightarrow\infty} \int_1^b x^{\alpha}\, dx
 =\lim_{b\rightarrow\infty}{  {x^{\alpha+1}\over \alpha+1} } \Bigl|_1^b  
 =\lim_{b\rightarrow\infty}\Bigl[{  {b^{\alpha+1}\over \alpha+1}  - {1\over \alpha+1}  }\Bigr].  
}
$$ 
To evaluate the limit in the right hand side of $(1)$, we   consider two cases: 
Case 1) $\alpha>-1$.
In this case, the exponent $\alpha+1>0$; whence $\lim\limits_{b\rightarrow\infty} b^{1+\alpha}=\infty$. Thus, evaluating the limit appearing in the right hand side of  $(1)$:
$$
\lim_{b\rightarrow\infty}\Bigl[{  {b^{\alpha+1}\over \alpha+1}  - {1\over \alpha+1}  }\Bigr]=\infty;$$
and so, the improper integral $\int_1^\infty x^\alpha\,dx$  does not converge for $\alpha>-1$.
Case 2) $\alpha<-1$.
In this case, the exponent $\alpha+1<0$; whence $\lim\limits_{b\rightarrow\infty} b^{1+\alpha}=0$. Thus, evaluating the limit appearing in the right hand side of  $(1)$:
$$
\lim_{b\rightarrow\infty}\Bigl[{  {b^{\alpha+1}\over \alpha+1}  - {1\over \alpha+1}  }\Bigr]= {-1\over \alpha+1};$$
and so, the improper integral $\int_1^\infty x^\alpha\,dx$ converges to $-1\over 1+\alpha  $ for $\alpha<-1$.

Summarizing what we have done: the improper integral
$$
\tag{2} \int_1^\infty x^\alpha\,dx
$$
 converges if and only if $\alpha<-1$.

Note that the improper integral in $(2)$  is just a "$p$-integral" which is conventionally   written
$$
\tag{3}\int_1^\infty {1\over x^p}\,dx
$$
The improper integral $(3)$ converges if and only if $p>1$.

For part a), assuming that you meant to write $\int_0^1 x^\alpha\,dx$, you need to recognize that this is also an improper integral when $\alpha<0$. In this case, the $y$-axis is a vertical asymptote of the graph of $y=x^\alpha$, and to determine if the integral converges, you would compute:
$$
\lim_{a\rightarrow0^+} \int_a^1 x^\alpha\,dx.
$$
