Can I apply Fundamental Theorem of Calculus here? Something tells me I can't, but I also feel I can because the integral is going to be messy if I integrate directly with trig substitution. 
The integral comes from a physics problem I am doing.
$$V(x) = k\lambda\int_{0}^{a} \dfrac{ d\ell}{\sqrt{x^2 + \ell ^2}}$$
$a$ is a constant, but I can let change $a$ to a variable like $h$
I am trying to find $-\dfrac{dV}{dx}$
The only thing that's bothering me is that x inside the integrand and setting $\ell = x$ after I apply chain rule would get me a $\sqrt{2}$ and I don't think the answer is going to come out this nice.
 A: You will find everything you need this article about differentiating under the integral sign. The article also deals with situations in which the limits of integration are not constant. 
The derivative $\frac{dV}{dx}$ is equal to the integral from $0$ to $a$ of the partial derivative (with respect to $x$) of your function.  In our case we get
$$\frac{dV}{dx}=-k\lambda\int_0^a \frac{x}{(x^2+\ell^2)^{3/2}}d\ell.$$
The integration can be done, for example, by letting $\ell=x\tan\theta$.  You end up needing to integrate $\cos\theta$, which is easy. Or else one can use $\ell=x\sinh t$.
In this case, the original integral can be calculated explicitly, and then we can differentiate.  However, if we use a trigonometric substitution, we end up integrating $\sec\theta$, which is not as pleasant. A $\sinh$ substitution is nicer. 
One could also treat $a$ as a variable, and differentiate. That part is easy, but in order to find $\frac{dV}{dx}$, an integration is still needed at the end. 
A: The short answer is "No, you can't do it like that."
The fundamental theorem of calculus would read as follows. Let
$$
F(a)=-k\lambda\int_{\ell=0}^a\frac{d\ell}{\sqrt{x^2+\ell^2}}.
$$
Then FTC tells that
$$
\frac{dF}{da}=-k\lambda\frac{1}{\sqrt{x^2+a^2}}.
$$
In other words, your mistake was about mixing the roles of different variables. In its basic form FTC allows you to differentiate a function of the variable appearing as the upper limit (here $a$) by substituting its value at a given point in place of the variable of integration (here $\ell$). In your example the variable $x$ is in neither of those roles, so it should be treated as a parameter as in André's answer.
