Choosing a surface that makes the flux of F maximal, For a closed surface S in $R^3$, consider the flux of F, given by the usual flux integral.  For what choice of S will the flux be maximal?
So, I want to apply the divergence theorem and instead look at the triple integral of the divergence of the given vector field F (over a solid that is enclosed by S.)
Since the flux integral = the divergence integral, I can aim to maximize the divergence integral.
My computation of the divergence gives me -3($x^2 + y^2 + z^2 - \frac{5}{3}$), so this would be the integrand in the triple integral.
It sounds reasonable that in order to maximize this triple integral, I should maximize the integrand.  How do I do that?
My attempt was to make $x^2 + y^2 + z^2$  - $\frac{5}{3}$ < 0, since there's a factor of -3 to consider.  negative * negative will give me a positive integrand - which would be good for maximizing the flux of F.
Then this tells me that I should choose a sphere of radius $\sqrt{(\frac{5}{3})}$.
I carried out my work and it looks like I got the correct answer.
But I feel like I chose my sphere ...by luck.
How do I know for sure that I've maximized the integrand, simply by making $x^2 + y^2 + z^2$  - $\frac{5}{3}$ < 0?  Could I have done even better, achieving a better maximum?  I simply knew that this factor had to be < 0, since it was being multiplied by -3.
Thanks,
 A: There is a very simple underlying principle. You choose the surface $S$ so that the region $V$ it bounds is precisely all the points with $\text{div}\,\vec F \ge 0$. Doing so will give you the greatest possible value for the integral $\iiint_V \text{div}\,\vec F\,dV$. (If you do not enclose all of those points, you get a smaller integral, and if you enclose a region with points where $\text{div}\,\vec F<0$, this makes the integral smaller as well.)
This problem has a solution precisely when the region $\{(x,y,z): \text{div}\,\vec F \ge 0\}$ is a (closed) bounded region. If the region is unbounded, there is no optimal surface $S$.
A: As you've correctly pointed out, the only region of positive divergence is the ball centered at $(0,0,0)$ with $r<\sqrt{\frac{5}{3}}$, due to the symmetry in the coordinates. Thus, we can focus our attention on this area and look at how divergence varies with radius. For convenience, let's express this as a univariate function of $r$: $\mathrm{div}(r)=5-3r^2$, where $A(r),V(r)$ is the area and volume of the ball of radius $r$.
$$\mathrm{Flux}(r)=\int_0^r\mathrm{div}(z)A(z)dz=5V(r)-3\int_0^r z^2A(z)dz = \frac{20}{3}\pi r^3-\frac{12}{5}\pi r^5$$
We want to maximize $\mathrm{Flux}(r)$, so lets find the extremal points $r^*$:
$$r^*:=r:\frac{d}{dr}\mathrm{Flux}(r)=20\pi r^2-12\pi r^4=0 \implies r=\pm \sqrt{\frac{5}{3}}$$
Just to check, let's verify the curvature:
$$\frac{d^2}{dr^2}\mathrm{Flux}\left(\sqrt{\frac{5}{3}}\right)=40\pi\sqrt{\frac{5}{3}}-48\pi\left(\frac{5}{3}\right)^{3/2}<0$$
So, we have verified what you expected.
