Maximize lateral surface of a square pyramid in a sphere I have to solve the following problem.

Task-description: A sphere with the radius of $10$ contains a pyramid with the
  maximal lateral surface ($M_{max}$). Find $M_{max}$.


I need two terms. One of them is the following, but I don't find a second one.

 A: Let $S=(0,0,1)$, let $P$ be one of the base points, and let $\alpha:=\angle(OSP)$. Then the length of the ascending edges is $2\cos\alpha$, and the length of the base diagonal is $2\sin(2\alpha)$. It follows that the side walls of the pyramid are isosceles triangles with base $\sqrt{2}\sin(2\alpha)$ and height 
$$h=\sqrt{4\cos^2\alpha-\sin^2(2\alpha)/2}\ .$$
Therefore the total lateral surface $M$ is given by
$$M=4{1\over2}\sqrt{2}\sin(2\alpha)\, h=8\sin\alpha\ \cos^2\alpha\sqrt{2-\sin^2\alpha}\ .$$
Putting $\sin\alpha=:s$ we now have to maximize the function
$$f(s):=s(1-s^2)\sqrt{2-s^2}\qquad(0\leq s\leq 1)\ .$$
Doing the computations gives $s_{\rm opt}=\sqrt{1-1/\sqrt{2}}$.
A: Let $O$ be the midpoint of the line segment $AB$. (I use the same notation for the points as in your picture.)
You have correctly noticed that
$$h_a^2=h^2+\frac{a^2}4,$$
which is equation obtained from the right triangle $SNO$.
You can use equation
$$r^2=(h-r)^2+\frac{a^2}2$$
which you obtain from the right triangle $MNA$. (Just notice that the distance $|MN|$ is $h-r$.)
The above equation is equivalent to
$$\begin{align}
r^2&=h^2-2hr+r^2+\frac{a^2}2\\
0&=h^2-2hr+\frac{a^2}2\\
2hr&=h^2+\frac{a^2}2\\
\frac{a^2}2&=2hr-h^2=h(2r-h)
\end{align}$$
So you have $h_a^2=h^2+\frac{a^2}4=h^2+\left(hr-\frac{h^2}2\right)=hr+\frac{h^2}2$.
Maximizing $2ah_a$ is equivalent to maximizing
$$(ah_a)^2=(2hr-h^2)(2hr+h^2).$$
Or, if you denote $t=\frac{h}r$, you get
$$(ah_a)^2=r^4(2t-t^2)(2t+t^2),$$
which reduces the problem to maximizing $(2t-t^2)(2t+t^2)$.
BTW are you sure you want to maximize $2ah_a$ and not $2ah_a+a^2$? (The second one is the surface including the base. It seems that this would be a more difficult problem.)
Note that you can get $\frac{a^2}2=2hr-h^2=h(2r-h)$ also from triangle $SAS'$, where $S'$ is the point opposite to $S$; if you use right triangle altitude theorem.  (The triangle $SAS'$ is right triangle according to Thales's theorem. 
The hypotenuse $|SS'|=2r$ is divided into two parts of lengths $h$ and $2r-h$.) 

Added later:
The function $f(t)=(2t-t^2 )(2t+t^2)=t^2(4-t^2)$ attains maximal value
$f(t)=4$ for $t^2=2$, i.e. $t=\sqrt2$.
(See Wolframalpha. This can be also seen directly, by maximazing the function $t^2(4-t^2)$ -- perhaps with the substitution $s=t^2$ -- or by by applying AM-GM inequality as $t^2(4-t^2)\le \frac{t^2+(4-t^2)}2=2$.)
This corresponds to $(ah_a)^2=4r^2$, $ah_a=2r$ and the lateral surface
area $$M=2ah_a=4r.$$ We also have $h=tr=\sqrt2r$.

If we compare this with Christian Blatter's solution, he gets maximal value $f(s_{\rm opt})=\frac12$, see wolframalpha. This gives $M=8f(s_{\rm opt})=4$.
We can also notice that the optimal angle is $\frac\pi8$, since
$s_{\rm opt}=\sqrt{1-1/\sqrt{2}}=\sqrt{\frac{2-\sqrt2}{\sqrt2}}=\sin\frac\pi8$,
see Wikipedia.
