compute $\iint_Y F.N\ dS $ with Gauss The question is:
$$ \iint_Y F.N\ dS \quad Y=(x-z)^2+(y-z)^2=1+z^2,\ \  0\leq z\leq 1 \quad N=\text{pointing outward}
$$ $$ F=(y,x,1+x^2z)$$
Here is how I have tried to solve it: I tried to solve it with Gauss theorem like this, $\gamma=\text{upper lid}$, $\sigma=\text{lower lid} $
$$ \iint_{Y+\sigma+\gamma}F.N\ dS=\iiint_K \text{div}f\  dA \iff \iint_Y F.N\ dS=\iiint_k-\iint_\sigma-\iint_\gamma$$
$$ \iint_\sigma=-\pi, \iint_\gamma=2+\pi$$
But I have trouble to integrate this triple integral
$$ \iiint_kx^2 dxdydz$$
I don't know if I have been  calculating right so far, and how should I proceed? Any suggestion would be great, thanks
 A: You can use transformation $f(x,y,z)=(x+z,y+z,z)$. The jacobian of this transform is 1. You get:$$
\iiint_{k'}(x+z)^2dxdydz
$$
where $x^2+y^2-z^2=1$.
Using cylindric coordinates, you can transform it to:
$$
\int_{0}^{1}\int_0^{2\pi}\int_0^{\sqrt{1+z^2}}(\rho\cos\phi+z)^2\rho d\rho d\phi dz
$$
Does this help you?
A: Surface is $(x-z)^2+(y-z)^2 = 1 + z^2$
$\vec F = (y,x,1+x^2z)$
To apply divergence theorem, we close the surface with discs at $z = 0$ and at $z = 1$.
a) Flux through disc at $z = 1$,
Plugging in $z = 1$ in the equation of hyperboloid, we get the equation of the disc as
$(x-1)^2+(y-1)^2 = 2$
In polar coordinates, $x = 1 + \rho \cos\theta, y = 1+\rho \sin\theta, z = 1 \ (0 \leq \rho \leq \sqrt2, 0 \leq \theta \leq 2\pi)$
Outward normal vector is $(0, 0, 1)$.
So the integral to find flux becomes
$\displaystyle \int_0^{2\pi} \int_0^{\sqrt2} [1+(1+ \rho \cos\theta)^2] \  \rho \ d\rho \ d\theta = 5 \pi$
b) Flux through the disc at $z = 0$ is simply the area of the disc with negative sign as $\vec F \cdot (0,0,-1) = -1$. So it is $-\pi$ as you correctly calculated.
c) Now to find flux through the closed surface, apply divergence theorem.
$\nabla \cdot \vec F = x^2$ as you mentioned.
Now the region for triple integral is a hyperboloid which is difficult to calculate without a Jacobian. So as the other answer suggests,
use change of variable $x = u + w, y = v + w, z = w$. Its Jacobian is $1$.
So the transformed region is $u^2 + v^2 - w^2 = 1$. In cylindrical coordinates,
$u = \rho \cos \theta, v = \rho \sin \theta, w$
$0 \leq \rho \leq \sqrt{1+w^2}, 0 \leq w \leq 1, 0 \leq \theta \leq 2\pi $
Integrand $x^2 = (u+w)^2 = (\rho \cos \theta + w)^2$
So integral to find flux,
$\displaystyle \int_0^{2\pi} \int_{0}^{1} \int_0^{\sqrt{1+w^2}} (\rho \cos \theta + w)^2 \ \rho \ d\rho \ dw \ d\theta = \pi$.
d) So flux through hyperboloid surface $ = \pi - 5 \pi - (-\pi) = - 3 \pi$
