Too long for a comment.
Your steps are very correct. The problem is that none of the CAS I used has been able to compute the second antiderivative or definite integral.
Using your second solution (which is better for the series solution approach)
$$I = \frac{2}{3} \int_{0}^{\frac{\pi}{3}} \sqrt{x}\, \left(3^{3/4}-\tan ^{\frac{3}{2}}(x)\right)\, dx=\frac{4 \pi ^{3/2}}{9\ 3^{3/4}}- \frac{2}{3} \int_{0}^{\frac{\pi}{3}} \sqrt{x}\,\tan ^{\frac{3}{2}}(x)\, dx$$ Expanding as a series built around $x=0$
$$ \sqrt{x}\,\tan ^{\frac{3}{2}}(x)=\sum_{n=1}^\infty a_n\,x^{2n}$$ where the first coefficients (given by Wolfram Alpha) are
$$\left\{1,\frac{1}{2},\frac{29}{120},\frac{1693}{15120},\frac{4361}{8
6400},\frac{1321}{59136},\frac{6380159677}{653837184000},\cdots\right\}$$ give
$$I=\frac{4 \pi ^{3/2}}{9\ 3^{3/4}}- \frac{2}{3}\sum_{n=1}^\infty\frac {a_n}{2n+1}\left(\frac{\pi }{3}\right)^{2 n+1}$$ Computing
$$J_k=\frac{4 \pi ^{3/2}}{9\ 3^{3/4}}- \frac{2}{3}\sum_{n=1}^k \frac {a_n}{2n+1}\left(\frac{\pi }{3}\right)^{2 n+1}$$ Using only the coefficients given in the linked page $J_{10}=0.693568$ while numerical integration gives $I=0.693519$.
For sure, using more terms leads to better and better results.
Edit
Since we have the series expansion, we can easily build the $[2(n+1),2n]$ Padé approximant $P_n$ around $x=0$. They write
$$P_n=x^2 \, \frac{1+\sum _{k=1}^n \alpha_k\, x^{2 k}}{1+\sum _{k=1}^n \beta_k\, x^{2 k} }$$ whose errors are $O(x^{4(n+1)})$.
Just to give an idea, consider the norm
$$\Phi_n=\int_0^{\frac \pi 3}\Big[\sqrt{x}\,\tan ^{\frac{3}{2}}(x)-P_n\Big]^2\,dx$$ Starting from $n=1$, they are
$$\{7.60\times 10^{-5},3.45\times 10^{-10},1.95\times 10^{-11},4.77\times 10^{-16},2.40\times 10^{-21}\}$$
After factor decomposition, wa have
$$P_n=\sum_{k=1}^n A_k \frac {x^2}{x^2-r_k}\qquad \implies \qquad \int_{0}^{\frac{\pi}{3}}P_n\,dx=\sum_{k=1}^n A_k \left(\frac{\pi }{3}-\sqrt{r} \tanh ^{-1}\left(\frac{\pi }{3
\sqrt{r_k}}\right)\right)$$
Up to $n=4$, we have (with readicals) the exact values of the $r_k$. So, considering the simple $P_3$ for which the $\alpha_k$ are
$$\left\{-\frac{4307737}{124259668},-\frac{6720869017}{139419347496
},\frac{15215045947}{13012472432960}\right\}$$ and the $\beta_k$ are
$$\left\{-\frac{66437571}{124259668},-\frac{15712003321}{6970967374
80},\frac{10427397914831}{351336755689920}\right\}$$
Converted to decimals
$$\int_0^{\frac \pi 3} P_3\,dx=0.58824037\qquad \implies \qquad \color{red}{I=0.69351}853$$ while numerical integration leads to
$I=0.69351943$
