Justification for indefinite integration by substitution I know that if we want to compute a definite integral using substitution, we use $$\int_{a}^b f(\phi(t))\phi '(t)~dt = \int_{\phi(a)}^{\phi(b)}f(x)~dx\tag1$$
I know that this method can go from left to right or right to left.
If we want to compute an indefinite integral using substitution, we use
$$\int f(t)~dt = \int g(\phi(t))\phi '(t)~dt = G(\phi(t)) + C, \tag{2}$$
where $G$ is an antiderivative of $g$ and $C$ is the constant of integration.
For example, for evaluating functions like $\dfrac{1}{\sqrt{4 - x^2}}$ we can make the substitution $x = 2\sin \theta$ and find $dx$ in terms of $d\theta$. Substituting its value in the integral, it becomes computable. However, pretending that $\dfrac{dx}{d\theta}$ is a fraction and substituting the value of $dx$ in the integral just because it "works" is not very rigorous. Is there a rigorous statement for computing indefinite integrals that can't be expressed in the aforementioned form using substitution, just like we have for definite integrals, or can they be computed using $(2)$ in a way that I can't figure out?
 A: This is kinda late, and I see you've already accepted an answer, but I'll present an answer of my own in case it helps you (even if it is just a tad bit more) or anyone else who reads it in the future. 
The whole business of $u$ - substitution in introductory single variable calculus, treating the differentials $dx,dy,du$ etc. as fractions when they explicitly tell us they're not fractions; then telling us we can define things called differential forms- but only much later on in our mathematical journey, is something which has bothered me for a while too... which is why I'm writing this.
I'll be using some notation which temporarily might not be very pleasing but bear with me.  So first of all I'll define the primitive of a (continuous) function $f$, $prim(f)$, to denote a function whose derivative is $f$ (as opposed to the indefinite integral notation). i.e for every point $x \in$Domain($f$), $[prim(f)]'(x) = f(x)$. Now of course, there are infinitely many primitives, but let's just fix one and stick with it.
Next, I'll state and prove the indefinite integral version of the substitution rule, which you have labelled ($1$), using the terminology of primitives.
Theorem: Suppose $f$ and $g'$ are continuous functions. Then $prim[(f \circ g) g'] = [prim(f)]\circ g$.
The meaning of this statement is that the derivatives of both sides agree; so we'll show just that.
The derivative of the LHS is (by definition of primitive) simply $(f \circ g) g'$.
For the RHS, we use the chain rule in differentiating to get : $\left[\left(prim(f)\right)' \circ g\right] g'$, which once again, by definition of primitive simplifies to $(f \circ g) g'$.
So, we've proved the indefinite integral version of the substitution rule. Now, I'll state this in notation we're all more familiar with. 
If $f$ and $g'$ are continuous then 
$$ \int (f \circ g) g' = \left( \int f \right)\circ g$$ or 
$$ \int f(g(x))\ g'(x)dx = \left( \int f \right) (g(x))$$ where as you can see, the RHS of the bottom equation is pretty awkward to write down; it's supposed to mean $prim(f)$  evaluated at the point $g(x)$. This is why I personally prefer the top equation- it's a statement about equality of functions, so there's absolutely no need to bring in the dummy variable/ parameter of integration (of course in actual computations its convenient but not when writing statements). 
Now, that I've explained my terminology sufficiently, I'll attempt to explain how to go about answering your question without any weird differential manipulations.
Let $f$ and $g$ be functions defined by the rules $$f(x) =\frac{1}{\sqrt{4-x^2}},\  g(\theta) = 2\ sin(\theta) $$
Now, the Theorem above states that $prim[(f \circ g) g'] = [prim(f)]\circ g$. Since we want a formula for $prim(f)$, we can solve for it by composing with $g^{-1}$ on both sides: $prim(f) = prim[(f \circ g) g']\circ g^{-1}$
As you noted, the RHS is simpler to compute so let's do that. The first part is computing: 
\begin{align*}
prim[(f\circ g)g'](\theta) &= \int f(g(\theta))\  g'(\theta) d\theta \\
&= \int \frac{1}{\sqrt{4 - (2\  sin\theta)^2}} 2\ cos\theta \ d\theta \\
&= \int 1 \ d\theta \\
&= \theta
\end{align*}
So this function, at the point $\theta$, has the value $\theta$. Which means its simply the identity map, $Id$. So, we have that $prim(f) = prim[(f \circ g) g']\circ g^{-1} = (Id)\circ g^{-1} = g^{-1}$ where $g^{-1}(x) = arcsin(\frac{x}{2})$.
i.e. we have shown that $$prim(f)(x) = g^{-1}(x) = arcsin\left( \frac{x}{2}\right)$$... or in more common notation:
$$\int f(x)dx = \int \frac{1}{\sqrt{4 - x^2}}dx =  arcsin\left( \frac{x}{2}\right)$$
Notice how throughout this answer I've tried to make a clear distinction between the functions in question: $f, g, prim(f), g^{-1}$ etc. and their values: $f(x), g(\theta), prim(f)(x), g^{-1}(x)$ and so on. The key to this whole argument was being careful about the functions vs their values, what you're composing, and rearranging the equation in the Theorem to get a formula for $prim(f)$- there was no multiplying/ cancelling differentials at all.
Now, I'll elaborate a bit more on the general process.
Often times you see the following statement instead: 
"Let $u = g(x), du = g'(x)dx $. Then,
$$ \int f(g(x))\ g'(x)dx = \int f(u) du$$"
This statement strictly speaking isn't accurate because it says that if we differentiate both sides we get the same result, but that isn't the case because $(f \circ g)g'$ is certainly not equal to $f$.
So what's going on when this isn't a correct equation yet we somehow end up with the correct answer? Well the subtlety is that the final substitution after the integration process corresponds to a composition. This is best explained by the following simple example.
Suppose we're asked to find $\displaystyle \int 2x e^{x^2}dx$. Here's how the typical explanation proceeds: first set $u = x^2, du = 2x\ dx$. Then, we make use of the "rule" $\displaystyle \int f(g(x))\ g'(x)dx = \displaystyle \int f(u) du$ to write:
\begin{align}
\int 2x e^{x^2}dx &= \int e^u du \\
&= e^u \\
&= e^{x^2} \\
\end{align}
And thus we've arrived at our answer. Now let's break down what's going on at each stage of the answer.
First, when people write "set $u = x^2$" what's really going on is they've identified that first of all the integrand is of the type $f(g(x)) g'(x)$. Next they're saying that $g(x) = x^2$ and that $f(x) = e^x$ 
Next, we have the first equality : $\int 2x e^{x^2}dx = \int e^u du$. This is the 'dubious equality' in the sense that these aren't strictly equal as functions. To see this, note that the variable of integration can be anything we like, so we can call it $u$ or $x$ or $t$ etc. So we can write this 'equality' as $\int 2x e^{x^2}dx = \int e^x dx$. I've simply changed the variable of integration from $u$ to $x$ on the RHS. But this says for every $x,$ $e^{x^2} = e^x$, which is clearly false.  Let's leave this aside for now, I'll get back to why it isn't a huge problem later.
Then we proceed with the next equality: $\int e^u du = e^u$. This simply says $prim(f)(u) = e^u$, or suppressing the variable, we can write it as $prim(f) = exp$. This step is fine :)
Lastly,we substitute $u = x^2$ to conclude $e^u = e^{x^2}$. This is the subtle step, because by substituting $u = x^2$ back into the equation, what's really going on is that we have just composed with $g$  ! This is why the final answer always turns out correct. 
I'll summarise these remarks below:
$$\int 2x e^{x^2}dx = \int e^u du \tag{!}$$ this says $prim((f\circ g)g') = prim(f)$, which is false.
Next, we have the string of equalities:
$$ \int e^u du = e^u = e^{x^2}$$
In the first equality, we're simply computing $prim(f)$. In the second one, we're substituting $u = x^2$ back, which has the same effect as writing the composition $prim(f)\circ g$. 
So lastly the reason why it's "alright" for us to write the equation I labelled as (!) as an equality is because later on when we substitute $u = x^2$ we are actually composing with g... which means we've made use of the Theorem proven above. That's how the Theorem proved above is implicitly being used in such explanations of the $u$- substitution rule.
In the equation $prim[(f \circ g) g'] = [prim(f)]\circ g$, sometimes the LHS might be easier to compute, for an appropriately chosen $g$ (as the question you asked) and sometimes the RHS may be easier to compute (as the example I provided shows). So the substitution rule does work both ways. So the answer to your question 

"Is there a rigorous statement for solving indefinite integrals which can't be expressed in aforementioned form using substitution just like we have for definite integrals or can they be computed using (1) in a way that I can't figure out?"

is yes! Statements about indefinite integrals/anti-derivatives/primitives are really statements about derivatives(not definite integrals) in disguise. So you can use the substitution rule (whose proof relied on the chain rule) as I have stated it and solve for the side you want :)
I tried to make explicit,the steps which are usually implicit and not explained clearly. So I hope this is helpful for you, and others who may have the same doubts!
A: You ask a good question. In the example you mentioned, the first step is simply writing down an antiderivative in abstract form, which is
$$\tag 1 \int_0^x \frac{1}{\sqrt {4-t^2}}\,dt,\,\,|x|<2.$$
We know that $(1)$ is an antiderivative for the integrand by the FTC. Now consider the function $t=2\sin s.$ This function maps $(-\pi/2,\pi/2)$ to $(-2,2)$ nicely. By the first equation you wrote,
$$\tag 2 \int_0^x \frac{1}{\sqrt {4-t^2}}\,dt =  \int_0^{\arcsin x/2} \frac{2\cos s}{\sqrt {4-(2\sin s)^2}}\,ds.$$
After we realize that the denominator on the right equals $2\cos s,$ we get $(1) = \arcsin x/2,$ and we're done.
This is the justification behind the scenes, although after a while you'll be cancelling $dx$'s with the best (worst) of them.
A: Define $y=f(x)$ and $x=s(t)$. Compute $\dfrac{dy}{dt}$ using chain rule:
$$\dfrac{dy}{dt}=\dfrac{dy}{dx}\dfrac{dx}{dt}$$
Now, to reverse the process, compute 
$$\int \frac{dy}{dx}\dfrac{dx}{dt}dt$$
which by our assumptions is $$y=f(x)$$
Now, you are given $\dfrac{dy}{dx}$ and you have defined $x$ as a function of $t$, so you can find $\dfrac{dx}{dt}$. Your problem is to find $y=f(x)$. That's the justification you are looking for. Introducing the new parameter $t$ is done just to simplify the integral, and all follows from the chain rule. You can try using your functions in the above equations.
NOTE: In the integral, you can hack the notation and say
$$\int\dfrac{dy}{dx}\dfrac{dx}{dt}dt=\int\dfrac{dy}{dx}dx$$
Edit after the comment
Let me use your example here. You are given $\dfrac{dy}{dx}=\dfrac{1}{\sqrt{4-x^2}}$ and you want to find $y=f(x)$, i.e the function whose derivative is this. You introduce $x=2\sin\theta$. So you know $\dfrac{dx}{d\theta}=2\cos\theta$. Take it from here.
A: 
$$\int_{a}^b f(\phi(t))\phi '(t)~dt =\int_{\phi(a)}^{\phi(b)}f(x)~dx\tag1$$
$$\int f(t)~dt = \int g(\phi(t))\phi '(t)~dt = G(\phi(t)) + C\tag2$$


pretending that $\dfrac{dx}{d\theta}$ is a fraction just
because it "works" is not very rigorous. Is there a rigorous statement
for computing indefinite integrals that can't be expressed in the
aforementioned form using substitution, just like we have for definite
integrals, or can they be computed using $(2)$ in a way that I can't
figure out?

Your query sounds like the two theorems are not intrinsically the same, so let's specify them using matching notation, and in full:

*

*If $g'$ is integrable on $[a,b]$ and $f$ is integrable, and has an antiderivative, on $g[a,b],$ then $$\int_a^bf\big(g(x)\big)\ g'(x)\,\mathrm{d}x=\int_{g(a)}^{g(b)}f(u)\,\mathrm{d}u.\tag{✓1}$$

*On the set $D,$ if $g$ is differentiable, $f$ has an antiderivative $F$ on $g[D]$ and $u=g(x),$ then $$\int f(g(x))\ g'(x)\,\mathrm dx=\int f(u)\,\mathrm du;$$ furthermore, on each interval $i$ of $D,$ $$\int f(g(x))\ g'(x)\,\mathrm dx=F(g(x))+C_i.\tag{✓2}$$
Sidestepping the informal notation, indefinite integration by subsitution ( $✓2$) is straightforward to rigorously derive.
Proof
By the chain rule, on $D,$ $$f(u)=F'(u)=F'(g(x))=f(g(x))g'(x)).$$ Then, noting that a function's indefinite integral is literally its family of antiderivatives, the first result of $(✓2)$ immediately follows.
On each interval $i$ of $D,$ $$\int f(u)\,\mathrm du=F(u)+C=F(g(x))+C_i,$$ and the second result of $(✓2)$ immediately follows.
$(✓1)$ follows from the Fundamental Theorem of Calculus.

Example (with $g(x)=x^3):$ $$\int\frac{x^2}{x^6+1}\,\mathrm dx=\int\frac1{3\Big(\color{violet}{(x^3)}^2+1\Big)}\color{cyan}{(3x^2)}\,\mathrm dx,$$ so the final result is $$\frac13\arctan\color{violet}{(x^3)}+C$$ while the intermediate result is $$\int\frac1{3(u^2+1)}\,\mathrm du.$$

Finally, a small disagreement with peek-a-boo's answer:

Often times you see the following statement instead:  "Let $u = g(x).$ Then $$ \int f(g(x))\ g'(x)\,\mathrm dx = \int f(u) \,\mathrm du"\tag A$$
This statement strictly speaking isn't accurate because if we
differentiate both sides we don't get the same result, because $(f
\circ g)g'$ is certainly not equal to $f.$
Suppose we're asked to find $\displaystyle \int 2x e^{x^2}\,\mathrm dx.$ First set $u = x^2$
....$$\int 2x e^{x^2}\,\mathrm dx = \int e^u \,\mathrm du."\tag B$$ This is the 'dubious equality'
in the sense that these aren't strictly equal as functions. To see
this, note that we can simply change the variable of integration from
$u$ to $x$ on the RHS.

No: $u$ is a dummy (i.e., bound) variable in the integral $\displaystyle\int_3^7 e^u \,\mathrm du$ but not in an indefinite integral, like $(\mathrm B)$'s $\displaystyle\int e^u \,\mathrm du,$ as it does not disappear from the indefinite integral's result.
Similarly, in $(\mathrm A),$ $$\int f(u) \,\mathrm du\not\equiv \int f(x) \,\mathrm dx,\\\frac{\mathrm d}{\mathrm dx}\int f(u) \,\mathrm du\not\equiv f(x),$$ so differentiating both sides of $(\mathrm A)$ certainly does not return the generally false equation $\displaystyle f(g(x))\ g'(x)=f(x).$
As long as its differentiability and antidifferentiability conditions (see $(✓2)$) are satisfied, statement $(A)$ is perfectly accurate!
