Definite integral of arcsine over square-root of quadratic 
For $a,b\in\mathbb{R}\land0<a\le1\land0\le b$, define $\mathcal{I}{\left(a,b\right)}$ by the integral
  $$\mathcal{I}{\left(a,b\right)}:=\int_{0}^{a}\frac{\arcsin{\left(2x-1\right)}\,\mathrm{d}x}{\sqrt{\left(a-x\right)\left(b+x\right)}}.\tag{1}$$

The integral $(1)$ above has closed forms in the following special cases:
$$\mathcal{I}{\left(a,0\right)}=4\,\chi_{2}{\left(\sqrt{a}\right)}-\frac{\pi^2}{2};~~~\small{0\le a\le1},\tag{2}$$
$$\mathcal{I}{\left(1,b\right)}=4\,\chi_{2}{\left(\frac{1}{\sqrt{1+b}}\right)}-\pi\operatorname{arccot}{\left(\sqrt{b}\right)};~~~\small{0<b},\tag{3}$$
where $\chi_{2}{\left(z\right)}$ is the Legendre chi function of order 2, which may be defined as
$$\chi_{2}{\left(z\right)}:=\int_{0}^{z}\frac{\operatorname{arctanh}{\left(t\right)}}{t}\,\mathrm{d}t=\frac{\operatorname{Li}_{2}{\left(z\right)}-\operatorname{Li}_{2}{\left(-z\right)}}{2};~~~\small{\left[\left|z\right|\le1\right]}.$$
I was also able to find a closed form for the special case where $b=a$ involving ${_4F_3}$ generalized hypergeometric functions. This leads me to believe that if the integral $\mathcal{I}{(a,b)}$ possesses a closed form at all, it will likely be in terms of hypergeometrics instead of simpler functions like the standard polylogarithms and elliptic integrals.

Question: Can $(1)$ be evaluated in terms of familiar special functions in the general case where $0<a<1\land0<b$? If not, can we at least find a nice hypergeometric function representation?


 A: 
For $0<a<1\land0<b$,
  $$\small{\mathcal{I}{\left(a,b\right)}=-\pi\arcsin{\left(\sqrt{\frac{a}{a+b}}\right)}+\frac{\pi a}{\sqrt{a+b}}F_{3}{\left(\frac12,\frac12;\frac12,\frac12;2;a,\frac{a}{a+b}\right)}.}$$


Proof:
The inverse sine function may be defined on the complex unit circle via the integral representation
$$\arcsin{\left(z\right)}:=\int_{0}^{z}\frac{\mathrm{d}t}{\sqrt{1-t^{2}}};~~~\small{z\in\mathbb{C}\land\left|z\right|\le1}.$$
Using the integral representation of $\arcsin{\left(z\right)}$ above, we may express $\mathcal{I}{\left(a,b\right)}$ equivalently as a double integral with an algebraic integrand. It has come to my attention that the resulting double integral has a natural expression an Appell hypergeometric function of the third kind:
$$F_{3}{\left(\alpha,\alpha^{\prime};\beta,\beta^{\prime};\gamma;z,w\right)}=\sum_{n=0}^{\infty}\sum_{k=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\alpha^{\prime}\right)_{k}\left(\beta\right)_{n}\left(\beta^{\prime}\right)_{k}}{\left(\gamma\right)_{n+k}n!\,k!}z^{n}w^{k},$$
where $\left|z\right|<1\land\left|w\right|<1$.
An integral representation for $F_{3}$ is derived in the appendix below.
Now, assuming $0<a<1\land0<b$, we find
$$\begin{align}
\mathcal{I}{\left(a,b\right)}
&=\int_{0}^{a}\frac{\arcsin{\left(2t-1\right)}}{\sqrt{\left(a-t\right)\left(b+t\right)}}\,\mathrm{d}t\\
&=\int_{0}^{a}\mathrm{d}t\,\frac{1}{\sqrt{\left(a-t\right)\left(b+t\right)}}\int_{\frac12}^{t}\mathrm{d}u\,\frac{1}{\sqrt{u\left(1-u\right)}}\\
&=-\int_{0}^{a}\mathrm{d}t\,\frac{1}{\sqrt{\left(a-t\right)\left(b+t\right)}}\int_{0}^{\frac12}\mathrm{d}u\,\frac{1}{\sqrt{u\left(1-u\right)}}\\
&~~~~~+\int_{0}^{a}\mathrm{d}t\,\frac{1}{\sqrt{\left(a-t\right)\left(b+t\right)}}\int_{0}^{t}\mathrm{d}u\,\frac{1}{\sqrt{u\left(1-u\right)}}\\
&=-\frac{\pi}{2}\int_{0}^{a}\frac{\mathrm{d}t}{\sqrt{\left(a-t\right)\left(b+t\right)}}\\
&~~~~~+\int_{0}^{a}\mathrm{d}t\,\frac{1}{\sqrt{\left(a-t\right)\left(b+t\right)}}\int_{0}^{t}\mathrm{d}u\,\frac{1}{\sqrt{u\left(1-u\right)}}\\
&=-\frac{\pi}{2}\int_{0}^{a}\frac{\mathrm{d}u}{\sqrt{u\left(a+b-u\right)}};~~~\small{\left[a-t=u\right]}\\
&~~~~~+\int_{0}^{1}\mathrm{d}x\,\frac{a}{\sqrt{ax\left(a+b-ax\right)}}\int_{0}^{a-ax}\mathrm{d}u\,\frac{1}{\sqrt{u\left(1-u\right)}};~~~\small{\left[\frac{a-t}{a}=x\right]}\\
&=-\frac{\pi}{2}\int_{0}^{\frac{a}{a+b}}\frac{\mathrm{d}v}{\sqrt{v\left(1-v\right)}};~~~\small{\left[u=\left(a+b\right)v\right]}\\
&~~~~~+\int_{0}^{1}\mathrm{d}x\,\frac{a}{\sqrt{ax\left(a+b-ax\right)}}\int_{0}^{1-x}\mathrm{d}y\,\frac{a}{\sqrt{ay\left(1-ay\right)}};~~~\small{\left[u=ay\right]}\\
&=-\pi\int_{0}^{\sqrt{\frac{a}{a+b}}}\frac{\mathrm{d}w}{\sqrt{1-w^2}};~~~\small{\left[\sqrt{v}=w\right]}\\
&~~~~~+\frac{a}{\sqrt{a+b}}\int_{0}^{1}\mathrm{d}x\int_{0}^{1-x}\mathrm{d}y\,\frac{1}{\sqrt{yx\left(1-ay\right)\left(1-\frac{a}{a+b}x\right)}}\\
&=-\pi\arcsin{\left(\sqrt{\frac{a}{a+b}}\right)}+\frac{\pi a}{\sqrt{a+b}}F_{3}{\left(\frac12,\frac12;\frac12,\frac12;2;a,\frac{a}{a+b}\right)}.\\
\end{align}$$

Appendix:
Given the parameter assumptions
$$\small{z,w\in\mathbb{D}\land\alpha,\alpha^{\prime},\beta,\beta^{\prime},\gamma\in\mathbb{C}\setminus\mathbb{Z}^{\le0}\land0<\Re{\left(\beta\right)}\land0<\Re{\left(\beta^{\prime}\right)}\land0<\Re{\left(\gamma-\beta-\beta^{\prime}\right)}}$$
an integral representation for the Appell hypergeometric function of the third kind may be derived as follows:
$$\begin{align}
F_{3}
&=F_{3}{\left(\alpha,\alpha^{\prime};\beta,\beta^{\prime};\gamma;z,w\right)}\\
&=\sum_{n=0}^{\infty}\sum_{k=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\alpha^{\prime}\right)_{k}\left(\beta\right)_{n}\left(\beta^{\prime}\right)_{k}}{\left(\gamma\right)_{n+k}n!\,k!}z^{n}w^{k}\\
&=\sum_{n=0}^{\infty}\sum_{k=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\alpha^{\prime}\right)_{k}\left(\beta\right)_{n}\left(\beta^{\prime}\right)_{k}}{\left(\gamma\right)_{n}\left(\gamma+n\right)_{k}n!\,k!}z^{n}w^{k}\\
&=\sum_{n=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\beta\right)_{n}z^{n}}{\left(\gamma\right)_{n}n!}\sum_{k=0}^{\infty}\frac{\left(\alpha^{\prime}\right)_{k}\left(\beta^{\prime}\right)_{k}w^{k}}{\left(\gamma+n\right)_{k}\,k!}\\
&=\sum_{n=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\beta\right)_{n}z^{n}}{\left(\gamma\right)_{n}n!}\,{_2F_1}{\left(\alpha^{\prime},\beta^{\prime};\gamma+n;w\right)}\\
&=\small{\sum_{n=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\beta\right)_{n}z^{n}}{\left(\gamma\right)_{n}n!}\cdot\frac{1}{\operatorname{B}{\left(\beta^{\prime},\gamma+n-\beta^{\prime}\right)}}\int_{0}^{1}\frac{t^{\beta^{\prime}-1}\left(1-t\right)^{\gamma+n-\beta^{\prime}-1}}{\left(1-wt\right)^{\alpha^{\prime}}}\,\mathrm{d}t}\\
&=\small{\sum_{n=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\beta\right)_{n}z^{n}}{\left(\gamma\right)_{n}n!}\cdot\frac{\Gamma{\left(\gamma+n\right)}}{\Gamma{\left(\beta^{\prime}\right)}\,\Gamma{\left(\gamma+n-\beta^{\prime}\right)}}\int_{0}^{1}\frac{t^{\beta^{\prime}-1}\left(1-t\right)^{\gamma+n-\beta^{\prime}-1}}{\left(1-wt\right)^{\alpha^{\prime}}}\,\mathrm{d}t}\\
&=\small{\frac{\Gamma{\left(\gamma\right)}}{\Gamma{\left(\beta^{\prime}\right)}\,\Gamma{\left(\gamma-\beta^{\prime}\right)}}\sum_{n=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\beta\right)_{n}z^{n}}{\left(\gamma-\beta^{\prime}\right)_{n}n!}\int_{0}^{1}\frac{t^{\beta^{\prime}-1}\left(1-t\right)^{\gamma+n-\beta^{\prime}-1}}{\left(1-wt\right)^{\alpha^{\prime}}}\,\mathrm{d}t}\\
&=\small{\frac{1}{\operatorname{B}{\left(\beta^{\prime},\gamma-\beta^{\prime}\right)}}\int_{0}^{1}\mathrm{d}t\,\frac{t^{\beta^{\prime}-1}\left(1-t\right)^{\gamma-\beta^{\prime}-1}}{\left(1-wt\right)^{\alpha^{\prime}}}\sum_{n=0}^{\infty}\frac{\left(\alpha\right)_{n}\left(\beta\right)_{n}z^{n}\left(1-t\right)^{n}}{\left(\gamma-\beta^{\prime}\right)_{n}n!}}\\
&=\small{\frac{1}{\operatorname{B}{\left(\beta^{\prime},\gamma-\beta^{\prime}\right)}}\int_{0}^{1}\mathrm{d}t\,\frac{t^{\beta^{\prime}-1}\left(1-t\right)^{\gamma-\beta^{\prime}-1}}{\left(1-wt\right)^{\alpha^{\prime}}}{_2F_1}{\left(\alpha,\beta;\gamma-\beta^{\prime};z\left(1-t\right)\right)}}\\
&=\frac{1}{\operatorname{B}{\left(\beta^{\prime},\gamma-\beta^{\prime}\right)}\cdot\operatorname{B}{\left(\beta,\gamma-\beta-\beta^{\prime}\right)}}\times\\
&~~~~~\int_{0}^{1}\mathrm{d}t\,\frac{t^{\beta^{\prime}-1}\left(1-t\right)^{\gamma-\beta^{\prime}-1}}{\left(1-wt\right)^{\alpha^{\prime}}}\int_{0}^{1}\mathrm{d}u\,\frac{u^{\beta-1}\left(1-u\right)^{\gamma-\beta-\beta^{\prime}-1}}{\left(1-z\left(1-t\right)u\right)^{\alpha}}\\
&=\frac{\Gamma{\left(\gamma\right)}}{\Gamma{\left(\beta\right)}\,\Gamma{\left(\beta^{\prime}\right)}\,\Gamma{\left(\gamma-\beta-\beta^{\prime}\right)}}\times\\
&~~~~~\small{\int_{0}^{1}\mathrm{d}t\,\frac{t^{\beta^{\prime}-1}}{\left(1-wt\right)^{\alpha^{\prime}}}\int_{0}^{1-t}\mathrm{d}y\,\frac{y^{\beta-1}\left(1-t-y\right)^{\gamma-\beta-\beta^{\prime}-1}}{\left(1-zy\right)^{\alpha}}};~~~\small{\left[\left(1-t\right)u=y\right]}\\
&=\frac{\Gamma{\left(\gamma\right)}}{\Gamma{\left(\beta\right)}\,\Gamma{\left(\beta^{\prime}\right)}\,\Gamma{\left(\gamma-\beta-\beta^{\prime}\right)}}\times\\
&~~~~~\int_{0}^{1}\mathrm{d}x\int_{0}^{1-x}\mathrm{d}y\,\frac{x^{\beta^{\prime}-1}y^{\beta-1}\left(1-x-y\right)^{\gamma-\beta-\beta^{\prime}-1}}{\left(1-wx\right)^{\alpha^{\prime}}\left(1-zy\right)^{\alpha}}.\blacksquare\\
\end{align}$$

A: Again, this is not going to be a full answer but since this problem is very similar to A tough series related with a hypergeometric function with quarter integer parameters I would like to sketch my approach. As usual in these kind of questions the tactic is to squeeze out certain parts that are given through elementary functions of Euler sums and then classify the remaining part and leave it un-evaluated for the time being. Denote $b_0=-b$. Then we have:
\begin{eqnarray}
&&-\imath {\mathcal I}(a,b_0)= \int\limits_0^a \frac{\arcsin(2x-1)}{\sqrt{(a-x)(b-x)}} dx=\\
&&\mbox{arcsin}(1-2 a) \log (b-a)-\pi  \log \left(\sqrt{a}+\sqrt{b}\right)+2 \int\limits_0^a \frac{\log\left[\sqrt{a-x}+\sqrt{b-x}\right]}{\sqrt{(1-x)x}}dx=\\
&&\mbox{arcsin}(1-2 a) \log (b-a)-\pi  \log \left(\sqrt{a}+\sqrt{b}\right)+2 \int\limits_0^a \frac{1/2\log\left[a-x\right]+\log\left[1+\sqrt{\frac{b-x}{a-x}}\right]}{\sqrt{(1-x)x}}dx=\\
&&\mbox{arcsin}(1-2 a) \log (b-a)-\pi  \log \left(\sqrt{a}+\sqrt{b}\right)+2 \log(a) \mbox{arcsin}(\sqrt{a})+2 \sqrt{a} {\mathcal J}(a)+\\
&&4(a-b) \int\limits_0^{\frac{\sqrt{a}}{\sqrt{a}+\sqrt{b}}}
\frac{(1-u)u}{1-2 u}\cdot \frac{\log(u)}{\sqrt{\left(a (u-1)^2-b u^2\right) \left(u^2 (b-a)+(2 a-2) u-a+1\right)}}du
\end{eqnarray}
In the top line we took out the minus out of the square root in the denominator. In the second line we integrated by parts and in the third line we factored the expression in the log appropriately. In the fourth line we split the remaining integral into two parts and we expressed the first part via the following function:
\begin{eqnarray}
{\mathcal J}(a)&:=& -\sum\limits_{l=0}^\infty a^l \binom{l-1/2}{l} \frac{H_{1/2+l}}{2l+1} \\
&=& \int\limits_0^1 \frac{\log[1-x^2]}{\sqrt{1-a x^2}} dx\\
&=&-\frac{1}{\sqrt{a}} \sum\limits_{\xi=\pm} \int\limits_{\pi/2}^{\arccos(\sqrt{a})} \log(1+\frac{\xi}{\sqrt{a}} \cos(\theta)) d\theta\\
&=&\frac{\imath}{6 \sqrt{a}} \left[
\pi^2+3 \imath \pi \log(4 a) - 6 \arccos(\sqrt{a})\left( \pi-\arccos(\sqrt{a})+\imath \log(4 a)\right)+
6 \sum\limits_{\xi_1=\pm,\xi_2=\pm} Li_2(\xi_1 \sqrt{1-a}+\imath \xi_1 \sqrt{a})
-6 \sum\limits_{\xi_1=\pm} Li_2(\xi_1(1-2 a-2 \imath \sqrt{a(1-a)}))
\right]
\end{eqnarray}
and in the second part we substituted for $u:= (1+\sqrt{(b-x)/(a-x)})^{-1}$.
We found the closed form for the function ${\mathcal J}(a)$ above using Anti-derivative of a function containing a log and a sine. .
Now the last integral is clearly very similar to the one in my answer to A tough series related with a hypergeometric function with quarter integer parameters. The integrans is a product of a rational function , a log and an inverse square root of a fourth order polynomial. Now, if there was no logarithm in the integrand it could have been always reduced to elliptic functions by following the recipe from here http://mathworld.wolfram.com/EllipticIntegral.html . However the logarithm makes thinks harder. Yet still maybe the methods for the Wolfram's site above make give an incling how to tackle such integrals.
