Why is the Hilbert space of Hilbert Schmidt operators important? The Hilbert Schmidt-operators form a Hilbert space with the inner product $\langle A, B \rangle = \text{tr}(B^*A)$.  What is the applications of this space?  What does it mean for $A$ and $B$ to be orthogonal?
 A: At the turn of the 20th century, Hilbert was just defining an abstract inner product space, the first and primary example being $\ell^2(\mathbb{N})$ consisting of sequences $\{ a_n \}_{n=0}^{\infty}$ for which $\sum_{n=0}^{\infty}|a_n|^2 < \infty$. It was natural to try to define an infinite-dimensional version of a matrix. Schmidt, who was a student of Hilbert's, looked at matrices $\{ m_{j,k} \}$ for which $\sum_{j,k}|m_{j,k}|^2 < \infty$. These are Hilbert-Schmidt operators. Keep in mind that general linear operators at not yet been defined at that point; this was part of that process leading to abstract linear spaces and linear operators. This class turned out to be useful because many of the integral solution operators coming from Physics were in the class.
I would say that orthogonality with respect to the Hilbert-Schmidt inner product is no more or less simple to interpret than a condition such as $\int_{a}^{b}f(t)\overline{g(t)}dt=0$, which is why it took a century after Fourier's work on "orthogonal function expansions" to lead to the definition of an inner product. An abstract inner product is useful for dealing with orthogonal expansions, just as it was for dealing with orthogonal function expansions. The inner product simply allows ideas of Euclidean geometry to be brought to the subject.
As an example, consider a Sturm-Liouville problem that would come up in separation of variables on a finite interval $[a,b]$:
$$
         Lf = -\frac{1}{w}\frac{d}{dx}\left[p\frac{df}{dx}\right]+qf
$$
where, in the simplest case, one has a continuous $w > 0$ on $[a,b]$, a continuously differentiable $p > 0$ on $[a,b]$, and a continuous $q$ on $[a,b]$. In this case, adding endpoint conditions of the following form leads to a resolvent operator for $(L-\lambda I)^{-1}$ that is a Hilbert-Schmidt operator:
$$
          \cos\alpha f(a)+\sin\alpha f'(a)=0,\\
          \cos\beta f(b)+\sin\beta f'(b)=0.
$$
These are common types of operators that occur when dealing with separation of variables problems for PDES on finite regions. As I mentioned, the resolvent operator is a Hilbert-Schmidt operator for all $\lambda$ that are not eigenvalues. The theory of Hilbert-Schmidt operators then gives the existence of an orthonormal basis of eigenfunctions of $L$ in the weighted Hilbert space $L^2_{w}[a,b]$. This is one of the earliest, powerful results of spectral theory.
