compact subspaces of the space $M_n(\mathbb{R})$ Let $M_n(\mathbb{R})$ denote the set of all $n \times n$ matrices with real entries, considered as the space $\mathbb{R}^{n^2}$.
 Which of the following subsets are compact?
(a) The set of all invertible matrices.
(b) The set of all orthogonal matrices.
(c) The set of all matrices whose trace is zero
 A: Hint
A set is compact in a finite dimensional normed space iff it's closed and bounded.
(a) Show using the determinant function that $\operatorname{GL}_n(\mathbb R)$ is open.
(c) The sequence of  matrices ($\operatorname {diag}(p,-p,0\ldots,0))_p$ isn't bounded.
A: (b) The set of all orthogonal matrices is compact.
To prove this we can use Sami's suggestion.  In particular, we want to show that 
the set $O_n =\{ A \in M_n(\mathbb{R}): AA^T = A^TA = I \}$ is closed and bounded as a 
subset of $\mathbb{R}^{n^2}$.
i) $O_n$ is closed:  Just take $\{ A_k \} \subset O_n$ such that $A_k \to A$ as $k \to \infty$, then clearly ${A_k}^T \to A^T$ as $k \to \infty$.  Hence, since $I = A_k{A_k}^T$ for all $k$, we have that $AA^T = I$.  In other words, $A \in O_n$.
ii) $O_n$ is bounded:  I will leave this for you to do.  A hint is to consider the relation $AA^T = I$ and find a bound on the entries of $A$.
A: Since $M_n(\Bbb R)$, when considered as a vector space over $\Bbb R$, is of finite dimension $n^2$, any two norms on $M_n(\Bbb R)$ are equivalent in the sense that they generate the same topology, i.e., the same open sets, the same closed sets, the same compacta, etc.  Thus, in addressing these questions, we have some freedom in the choice of norm we make.  In what follows, I take the norm on $M_n(\Bbb R)$ to be that induced by the standard inner product $\langle \cdot, \cdot \rangle$ on $\Bbb R^n$, viz. $\langle x, y \rangle = \sum_1^n x_i y_i$ for $x = (x_1, x_2, . . ., x_n)^T$ etc. in the standard basis of $\Bbb R^n$.  Then $\Vert AB \Vert \le \Vert A \Vert \Vert B \Vert$ and $\Vert O \Vert = 1$ for any $A, B \in M_n(\Bbb R)$ and any orthogonal $O \in M_n(\Bbb R)$.  These well-known facts are exploited in what follows.
For (a), we show that $GL_n(\Bbb R) \subset M_n(R)$ is open.  Let $g \in GL_n(\Bbb R)$; we will show that for $h \in M_n(\Bbb R)$ sufficiently small in norm, $g + h \in GL_n(\Bbb R)$.  Observe that $g + h = g (I + g^{-1}h)$; consider $g^{-1}h$.  If we use any norm $\Vert \cdot \Vert$ on $M_n(\Bbb R)$ such that $\Vert AB \Vert \le \Vert A \Vert \Vert B \Vert$, choose $h \in M_n(\Bbb R)$ such that $\Vert h \Vert < \Vert g^{-1} \Vert^{-1}$; then $\Vert g^{-1}h \Vert \le \Vert g^{-1} \Vert \Vert h \Vert < 1$.  We conclude that for such $h$, $(I + g^{-1}h)$ is invertible, since the power series $I - g^{-1}h + (g^{-1}h)^2 - (g^{-1}h)^3 + . . .  = \sum_0^\infty (-g^{-1}h)^i$ converges absolutely by virtue of $\Vert g^{-1}h \Vert  < 1$; it is easy to see that $(I + g^{-1}h)^{-1} = \sum_0^\infty (-g^{-1}h)^i$.  Since $g + h = g(I + g^{-1}h)$, and $g^{-1}$ exists, so does the inverse of $g + h$; thus $g + h \in GL_n(\Bbb R)$.  We thus have $GL_n(\Bbb R)$ open, hence NOT COMPACT.
For (b), note that $Q^TQ = I$ for any orthogonal matrix $Q$; now let $O_m$ be a sequence of orthogonal matrices which converges in norm to some matrix $O$.  $O_m^TO_m = I$ for all $m$, so by continuity in norm of matrix multiplication, we have $O^TO = \lim_{m \to \infty} O_m^TO_m = I$; $O = \lim_{m \to \infty} O_m$ is orthogonal; thus we see the set of orthogonal matrices is closed in $GL_n(\Bbb R)$.  Since $\Vert Q \Vert = 1$ for any orthogonal $Q$, the set of orthogonals is also bounded; being closed and bounded, it is COMPACT.
For (c), observe that the map $Tr:M_n(\Bbb R) \to \Bbb R$ which takes $A  = [a_{ij}] \in M_n(\Bbb R)$ to $\sum_1^n a_{ii} \in \Bbb R$ is in fact linear over $\Bbb R$; furthermore, it is evident that $\ker Tr$ is precisely the matrices in $M_n(\Bbb R)$ with vanishing trace; see my answer to this question for further details.  Since $Tr$ is linear, $\ker Tr$ is a linear subspace of $M_n(\Bbb R)$, hence not bounded, hence NOT COMPACT.
