Give a formula for the volume of the solid under a surface $z=xy$ and a triangle? 
Give a formula for the volume of the solid with unit density lying under the surface $z = xy$ and above the triangle in the $xy$-plane with vertices $(0, 1, 0)$, $(1, 1, 0)$ and $(0, 2, 0)$.

What is the correct way to do this?
Attempt:
The volume $V$ lying beneath the surface $z = f(x,y)$ and above the region
$a\leq x\leq b,g_{1}(x)\leq y\leq g_{2}(x)$
$V=\iint_{D}f(x,y)dA=\int_{a}^{b}\int_{g_1(x)}^{g_2(x)}f(x,y)dydx$
In the triangle with vertices $(0, 1, 0)$, $(1, 1, 0)$ and $(0, 2, 0)$, $x$ lies between
$0$ and $1$
The $y$ component could lie between $y = 1$ and the line joining the points $(1, 1)$ and $(0,2)$. And it could have the equation $y = -x + 2$.
$D= \left \{ (x,y):0\leq x\leq 1,1\leq y\leq -x+2 \right \}$
$V=\iint_{D} (xy)dA=\int_{0}^{1}\int_{1}^{ -x+2}xydydx$
 A: Here's another way to get answer to this question:


*

*Rename the vertices: $A_1 = (0,1,0), A_2 = (1,1,0), A_3 = (0,2,0)$.

*Find area $$A_x = [\min{(A_{1x},A_{2x},A_{3x})} .. \max{(A_{1x},A_{2x},A_{3x})}] = [\min(0,1,0) .. \max(0,1,0)] = [0..1]$$
$$A_y = [\min{(A_{1y},A_{2y},A_{3y})} .. \max{(A_{1y},A_{2y},A_{3y})}] = [\min(1,1,2) .. \max(1,1,2)] = [1..2]$$
$A=[0..1]\times[1..2]$. (bounding rectangle)

*Find subset of A which is inside the triangle, $K \subset A$: 
$$ K = \{ (x,y) | isInsideTriangle(x,y) \wedge (x,y) \in A \}$$, where isInsideTriangle can be found from url https://stackoverflow.com/questions/2049582/how-to-determine-if-a-point-is-in-a-2d-triangle

*Note that $z=xy$ can be written in form $z-xy=0$ which is of form $f(x,y,z)=0$.

*Use implicit function and heightmap substitution $z=f(x,y)$: $$f(x,y,z) = f(x,y,f(x,y)) = 0$$
$$ f(x,y)-xy = 0$$, $$ f(x,y) = xy $$

*Calculate area of the triangle $$ A_{tri} = \frac{1}{2} A_x \cdot A_y = \frac{1}{2} 1 \cdot 1 = \frac{1}{2} $$

*Calculate average of f(x,y) by random sampling, utilizing the area $K$ and a sum $av_f(K) = \frac{\sum_{i=1}^{k} f(x_i,y_i)}{k}$, where $f(x_i,y_i)$ is random sample taken from $(x_i,y_i) \in K, i \in [1..k], k>10000$.

*Note that $\int_K{f(x,y)dK} = A_{tri} \times av_f(K) $ where $av_f(K)$ is the average of $f(x,y)$ in area $K$.
A: Your work is right. It is true that you can write the definition of the whole three dimensional region, say $W\subset \mathbb R^3$ and find
$$V(W)=\iiint_W\,dV,$$
but it's also fine to remember that since $W$ happens to be the region between the graph of a non-negative function, the $xy$ plane and the vertical projection of $T\subset \mathbb R^2$, then
$$V(W)=\iint_T f(x,y)\,dA.$$
In fact, double Riemann sums and double integrals are defined in such a way as to deliberately represent the volume of these kind of 3D regions.
And variables are fine, since you actually want that the $x$ of $f(x,y)$ to get 'mixed' with those in the limits of integration. That's the idea of iterated integrals: for instance, you first calculate the integral with respecto to variable $y$ given by
$$\int_{g_1(x)}^{g_2(x)} f(x,y)\,dy;$$
this gives a function of $x$ (resulting from the combination of treating the $x$ of $f(x,y)$ as a constant when you integrate for $y$, and from those in $g_1$ and $g_2$ which are replaced in the $y$ of the primitive when applying Barrow's formula, since the integral is in $y$ it's for the limits to depend on $x$). Then that function is integrated w.r.t. $x$, between two constant values (not dependent in $x$ nor in $y$), and this gives the number
$$\int_a^b\left(\int_{g_1(x)}^{g_2(x)} f(x,y)dy\right)\,dx.$$
The only thing you should avoid is integrating in a variable and having the limits of integration in the same variable. That is, you should not write
$$\color{darkred}{\int_0^{x^2}2x\cos(y)\,dx},$$
but there's nothing wrong with the expression
$$\color{darkgreen}{\int_0^{x^2}2x\cos(y)\,dy}.$$
So, good job!
A: What you did looks good providing some light modifications:


*

*The volume is a 3d integral. So I would start saying that your aim is to compute the volume V of the domain $\mathcal{V}=\{(x,y,z) \in \mathbb{R}^3 \ | 0\leq x\leq 1,1\leq y\leq -x+2, 0 \le z \le xy \}$.

*Then $V= \int \int \int_\mathcal{V} dx dy dz=\int \int_Dxy dx dy$.

*Then you have a small issue of using $x$ as both a bound of integration and a variable in the function to be integrated which is confusing.

