Given $f(x)=x^2+ax+2b$ with $a\in [1.2,5]$ and $b\in [-0.8,1.5]$, what is the probability that $f(x)$ has two real roots? A friend and me tried to solve this problem and we took two different approachs:
while both considered the probability as a quotient between areas, he used an auxiliary function
$$g(a)=\frac{a^2}{8}$$ and computed the following area:
Approach 1, getting probability as shadowed area over rectangle area $P\approx 0.801$.
On the other hand I took the two intervals $a^2\equiv x\in [1.2^2,5^2]$ and $8b\equiv y\in 8\cdot[-0.8,1.5]$ obtaining the following area Approach 2 and computating the probability as area below $y=x$ over rectangle area $P\approx 0.87$.
So, I have been trying to understand which one of the approachs is wrong and why but I have failed so I am here.
EDIT:
also, is there any point $(x\equiv a^2,y\equiv 8b)$ in the region where $x>y$ of the approach 2 that return a quadratic polynomial without two real roots?
 A: Counting roots of the equation $Ax^2 + Bx + C = 0$ depends on the sign of the determinant $B^2-4AC$:

*

*There are two real roots if and only if $B^2-4AC > 0$
For your problem, where $A=1$, $B=a$, and $C=2b$, this translates into:

*

*There are two real roots if and only if $a^2 - 8b > 0$, equivalently
$$b < \frac{a^2}{8}
$$
So the region in Approach 1 is indeed the correct region to consider. And as you said in the comments, $a$ and $b$ are independent and uniformly distributed, so the probability measure on that rectangle is indeed proportional to probability (you just have to divide out by the total area $(5 - 1.2) \times (1.5-(-.8))$).
I would like to be able to critique Approach 2, however the axes are not labelled and no region is shaded in, so it's hard to know what to say. Even assuming the vertical axis is $y$ and the horizontal axis is $x$ as you seem to be hinting in your post, I'm not sure what relation the equation $y=x$ has to the equation $x^2 + ax + 2b$.
A: In order to have two real roots, we require that $a^2-8b>0,$ where the ordered pair $\langle a,b\rangle$ is taken from some distribution on $\left[\frac65,5\right]\times\left[-\frac45,\frac32\right].$ I assume herein that this distribution is assumed to be uniform.
Put another way, since $a^2$ is necessarily positive, we require that $b<\frac18a^2,$ given $a,b$ in the selected area.
Geometrically, this amounts to the area under the curve $y=\frac18 x^2,$ under the line $y=\frac32,$ and over the line $y=-\frac45$ within the $x$-interval $\left[\frac65,5\right].$
After drawing the graphs of these curves, it's clear that $\frac18x^2>-\frac45$ for all $x.$ Moreover, $\frac18x^2=\frac32$ exactly when $x=\sqrt{12}$ for $x\in\left[\frac65,5\right].$ Hence, we're looking at the area between $y=\frac18 x^2$ and $y=-\frac45$ in the interval $\left[\frac65,\sqrt{12}\right],$ and between $y=\frac32$ and $y=-\frac45$ in the interval $\left[\sqrt{12},5\right].$
Put another way, we need to evaluate $$\int_{\frac65}^{\sqrt{12}}\left(\frac18x^2+\frac45\right)\,dx+\int_{\sqrt{12}}^5\left(\frac32+\frac45\right)\,dx$$ for the answer.
A: The issue with Approach 2 is the non-linearity of your variable transformation.  The problem this creates is that the area of the triangle in the upper-left corner of the graph, which represents the complementary probability, is made disproportionately small , so a simple subtraction of that triangular area leads to an over-estimate of the probability sought.

Let's carry out the integration for "Approach 1" first in order to compare it against what must be done for the other Approach.  We will compute the area "above" the parabola $ \ b \ = \ \frac{a^2}{8} \ \ , $ in order to determine the complement to the probability for two real zeroes.  The total area of the "population space" rectangle is $ \ \mathbf{A} \ = \ ( \ 5 - 1.2 \ )·( \ 1.5 - [-0.8] \ ) \ = \ 8.74 \ \ . $  The area above the parabolic arc is found from the double integral
$$ \mathcal{A} \ \ = \ \ \int_{1.2}^{\sqrt{12}} \int_{a^2/8}^{1.5} \ \ db \ \ da \ \ = \ \ \int_{1.2}^{\sqrt{12}} \int_{a^2/8}^{1.5} \ \ db \ \ da \ \ = \ \ \int_{1.2}^{\sqrt{12}} \ 1.5 \ - \ \frac{a^2}{8} \ \ \ da $$ $$ = \ \  \left. \left( \ 1.5·a \ - \ \frac{a^3}{24} \ \right)\right|_{1.2}^{\sqrt{12}} \ \ \approx \ \ 1.736 \ \ . $$
[The upper limit on the integral in $ \ a \ $ is found from $ \ \frac{a^2}{8} \ = \ 1.5 \ \ .  $ ]  We thereby calculate the probability that the given quadratic polynomial has two real zeroes in the specified "coefficient region" as
$$ 1 \ - \ \frac{\mathcal{A}}{\mathbf{A}} \ \ \approx \ \ 1 \ - \ \frac{1.736}{8.74} \ \ = \ \ 0.801 \ \ . $$

For the variable substitution $ \ u \ = \ a^2 \ \ , \ \ v \ = \ 8b \ \ , $ we obtain the rectangle above; the parabolic arc becomes $ \ b \ = \ \frac{a^2}{8} \ \rightarrow \ v \ = \ u \ \ . $  We will want to compute the complementary probability by finding the area of the triangle above that line and within the rectangle.  This requires us to transform the differentials in the integral as well, which we achieve by the use of the Jacobian
$$ \mathfrak{J} \ \ = \ \ \left[\begin{array}{cc}
\frac{\partial u}{\partial a} & \frac{\partial u}{\partial b} \\
\frac{\partial v}{\partial a} & \frac{\partial v}{\partial b} \end{array} \right] \ \ = \ \ \left[\begin{array}{cc}
2a & 0 \\
0 & 8 \end{array} \right] \ \ \Rightarrow \ \ | \ \mathfrak{J} \ | \ \ = \ \ 16a \ \rightarrow \ 16·\sqrt{u} \ \  .  $$
Under this transformation, even the area of the rectangle must be integrated and is not simply the product of its length and width.  We now calculate
$$ \mathbf{A'} \ \ = \ \ \int_{1.2^2}^{5^2} \int_{8·(-0.8)}^{8·1.5} \ \ \frac{1}{| \ \mathfrak{J} \ |} \ \ dv \ \ du \ \ = \ \ \int_{1.44}^{25} \int_{-6.4}^{12} \ \ \frac{1}{16·\sqrt{u}} \ \  dv \ \ du $$ $$ = \ \ \frac{1}{16} \ \int_{1.44}^{25} \ \left. \frac{v}{\sqrt{u}} \right|_{-6.4}^{12} \ \ \ du \ \ = \ \ \frac{18.4}{16} \ \int_{1.44}^{25} \   \frac{1}{\sqrt{u}}   \ \ \ du \ \ = \ \ \frac{18.4}{16} \ · \ 2  ·    \sqrt{u}|_{1.44}^{25} \ \ = \ \ 8.74 \ \ . $$
So we find the same area for the "transformed" rectangle as we did with the original coordinates.  The area of the transformed complementary region is
$$ \mathcal{A'} \ \ = \ \ \int_{1.2^2}^{\sqrt{12}^2} \int_{u}^{8·1.5} \ \ \frac{1}{16·\sqrt{u}} \ \  dv \ \ du \ = \ \ \frac{1}{16} \ \int_{1.44}^{12} \ \left. \frac{v}{\sqrt{u}} \right|_{u}^{12} \ \ \ du $$ $$ = \ \ \frac{1}{16} \ \int_{1.44}^{12} \  \left( \ \frac{12}{\sqrt{u}} \ - \ \sqrt{u} \ \right)   \ \ du \ \  = \ \  \frac{1}{16} \ \left. \left( \ 12·2·\sqrt{u} \ - \ \frac{2}{3}·u^{3/2} \ \right)\right|_{1.44}^{12} \ \ \approx \ \ 1.736 \ \ . $$
We find the area of the transformed region to be the same as that of the original one, giving us the probability we wished to determine as
$$ 1 \ - \ \frac{\mathcal{A'}}{\mathbf{A'}} \ \ = \ \ 1 \ - \ \frac{\mathcal{A}}{\mathbf{A}} \ \ \approx  \ \ 0.801 \ \ . $$
The discrepancy in Approach 2 is due to the omission of  the non-linear effect of the chosen variable transformation.  (Leaving out the Jacobian is a not uncommon error in such calculations.)
