Representation of $SU(2)$ Let
\begin{equation}
X= \begin{bmatrix}
0 & 1\\
0 & 0\\
\end{bmatrix}, \qquad
Y= \begin{bmatrix}
0 & 0\\
1 & 0\\
\end{bmatrix}, \qquad
H= \begin{bmatrix}
1 & 0\\
0 & -1\\
\end{bmatrix}
\end{equation}
If $V_m$ is the $(m+1)$-dimensional complex representation of $\text{sl}(2,\mathbb{C})$, then we know that there exists a basis $u_m,u_{m-2},...,u_{-m}$ such that $u_k$ is the eigenvector of $H$ with eigenvalue $k$ and $Y u_k = u_{k-2}$.
In physics, on the other hand, we write $S_{\pm} =X,Y$ and $S_z = H/2$ and use the basis $|s,m_s \rangle$ such that
\begin{align}
S_z |s,m_s \rangle &= m_s |s,m_s \rangle \\
S_{\pm} |s,m_s \rangle &= \sqrt{s(s+1)-m_s (m_s \pm 1)} |s,m_s \pm 1\rangle
\end{align}
and that $|s,m_s \rangle$ are orthogonal. 
Is there a deeper reason why we put the coefficients in front of $|s,m_s \rangle$, when $S_{\pm}$ acts on it and why $|s,m_s \rangle$ are orthogonal?
 A: The coefficients, say $\alpha_\pm =\sqrt{s(s+1)-m_s(m_s \pm 1)}$, arise algebraically in the move from the real, compact Lie algebra $\mathfrak{su}_2$ to its complexification $\mathfrak{su}_2 \otimes \mathbb{C} \cong \mathfrak{sl}_{2 \mathbb{C}}$, a complex, non-compact Lie algebra.
To see this, start by noting the commutation relations $$[S_z,S_\pm] = \pm S_\pm, \;\;\;\; [S_+, S_-]=S_z,$$ and rearrange to get $$S_z S_\pm = S_\pm S_z \pm S_\pm.$$
Since $\vert s, m_s \rangle$ represents an eigenvector of $S_z$, $S_z$ reduces to the eigenvalue $m_s$ in this basis, and $$S_z \, S_\pm \, \vert s, m_s \rangle = (S_\pm S_z \pm S_\pm) \vert s, m_s \rangle = (m_s \pm 1) \, S_\pm \, \vert s, m_s \rangle.$$
Next observe that $$S_z \vert s, m_s+1\rangle = (m_s+1) \vert s, m_s +1 \rangle$$ which combined with the above implies $$S_\pm \vert s, m_s \rangle = \alpha_\pm \vert s, m_s +1 \rangle,$$
where $\alpha_\pm$ is some scalar.
To calculate $\alpha_\pm$ in the context of angular momentum, take a step back and note that we created the ladder operators $S_\pm \in \mathfrak{sl}_{2\mathbb{C}}$ from unitary operators $S_x, \, S_y, \, S_z \in \mathfrak{su}_2$, satisfying $$[S_i,S_j] = i \, \epsilon_{ijk} \, S_k,$$
by $S_\pm = (S_x \pm i S_y)$.  Noting further that $S_+=S_-{}^\dagger$ we have
$$\langle s, m_s \vert S_\mp \, S_\pm \vert s, m_s \rangle = \langle s, m_s + 1 \vert \alpha_\pm^\ast \, \alpha_\pm \vert s, m_s+1 \rangle.$$
Using the definition of the ladder operators and the various commutators, we also have
$$S_\mp \, S_\pm = (S_x \mp i S_y)(S_x \pm i S_y) = S^2 - S_z{}^2 \pm S_z.$$
$S^2$ is a multiple of the unit matrix with $S^2 = s(s+1)$, and we are operating in an eigenbasis of $S_z$, so $S_z{}^2$ and $S_z$ reduce to their eigenvalues. Disregarding the phase of $\alpha_\pm$ as not physically relevant, we finally get
$$\alpha_\pm^\ast \alpha_\pm =\vert \alpha_\pm \vert^2 = s(s+1)-m_s{}^2 \pm m_s = s(s+1)-m_s(m_s \pm 1),$$
which is where your coefficients come from.
If you are looking for more depth on the underlying mathematics, I recommend $\textit{Representation Theory}$ by Fulton and Harris (specifically Lecture 11), which is an excellent reference to understand the deep mathematical theory only superficially addressed here.
If you are looking for more depth in a physical setting, I recommend Hermann Weyl's $\textit{The Theory of Groups and Quantum Mechanics}$ (specifically Chapter III, Section 15, where you'll find your equations almost verbatim and fully motivated).
Your second question is much easier, $S_z$ is normal, and every normal matrix has an orthogonal eigenbasis.
