Definite integral FTC Find a function f so that:
$\int _a^{x^2}\:f\left(t\right)ln\left(t\right)dt\:=\:x^3\left(ln\left(x\right)-\frac{1}{3}\right)$, $a>1$
This is how I did it:
First get the derivative of $x^3\left(ln\left(x\right)-\frac{1}{3}\right)$ which is $3x^2ln(x)$
Then the derivative of the integral which according to the theorem is: $2xf(x^2)ln(x^2) - f(a)ln(a)*a'$, but $a'=0$
Now solve for $f(x)$, $2xf(x^2)ln(x^2) = 3x^2ln(x)$
$f(x^2) =\frac{3x^2ln\left(x\right)}{2xln\left(x^2\right)}$
$f(x^2) = 3x/4$
That makes $f(x) = \frac{3\sqrt{x}}{4}$, right?
 A: Note that if you take $f(t) = \frac{g(a)}{(x^2-a)\log(t)}$ then 
$$
\int_{a}^{x^2} f(t)\log(t)dt = \int_{a}^{x^2} \frac{g(a)}{(x^2-a)}dt = g(a)
$$
Moreover, if you take the derivative of $g(a)$ with respect to $x$ you will get $0$, so that means that when you took derivatives of the left hand side and the right hand side you have to take into a account that you can add any term $\frac{g(a)}{(x^2-a)\log(t)}$ to the function $f(t)$ without the derivative with respect to $x$ of the LHS changing.  
That being said you did all the hard work.  Since Mathematica gives the integral 
$$
\int_0^x \sqrt{t}\log(t) dt = \frac{2}{9}x^{3/2}(3\log(x)-2)
$$
we can plug in $x^2$ into this expression to verify that 
$$
\int_0^{x^2} \frac{3}{4}\sqrt{t}\log(t) dt =x^3(\log(x)-\frac{1}{3}),$$ so your answer is the solution when $a=0$.
We can find the general solution by noting that
$$
\int_a^{x^2} \left(\frac{3}{4}\sqrt{t}+\frac{g(a)}{(x^2-a)\log(t)}\right) \log(t) dt = g(a)+x^3(\log(x)-\frac{1}{3})-a^{3/2}(\frac{1}{2}\log(a)-\frac{1}{3})
$$
The answer should be $x^3(\log(x)-\frac{1}{3})$ therefore we must have
$g(a) = a^{3/2}(\frac{1}{2}\log(a)-\frac{1}{3})$ which means that 
$$
f(t) = \frac{3}{4}\sqrt{t}+\frac{a^{3/2}(3\log(a)-2)}{6(x^2-a)\log(t)}
$$
You had the requirement $a>1$ in your problem statement.  It seems the statement holds for any $a\geq 0$ by my argument.
