# Invariant subspaces of Lie group vs invariant subspaces of Lie algebra

I am starting to study infinite-dimensional representations of Lie groups and I am wondering about the following:

Let $$G$$ be a connected Lie group with Lie algebra $$\mathfrak g$$ and with a representation $$\Phi: G \to \text{GL}(V)$$, with $$V$$ finite-dimensional. Then, $$\Phi$$ induces a unique representation $$\varphi: \mathfrak g \to \text{End}(V)$$ of the Lie algebra and a subspace of $$V$$ is $$G$$-invariant if and only if it is $$\mathfrak g$$-invariant.

I am struggling to see why and how exactly is this statement wrong for infinite-dimensional representations? I do not see where the proof does not carry over. Also, what are easy illustrative (counter-)examples and what does it mean for a Lie algebra representation to be integrable?

Edit: On a second thought, could it be that in the infinite-dimensional case, the function $$g \mapsto \Phi(g)v$$ will not be differentiable for all $$v \in V$$ and thus we can not that easily differentiate our Lie group representation to a Lie algebra representation? Then my additional question is: where does the proof that a continuous Lie group homomorphism is automatically smooth, does not carry over? The group $$\text{GL}(V)$$ then no longer is finite-dimensional which makes problems, I guess.

• $GL(V)$ is horrible if $V$ is infinite dimensional. There are many linear transformations that aren't even continuous. Commented Jul 16, 2015 at 4:24
• Yes, but the definition of a representation requires the operators $\Phi(g)$ to be linear and invertible. Also I have the impression that in the theory of representations, one only considers continuous representations. Commented Jul 16, 2015 at 5:10
• There are plenty of invertible linear transformations that aren't continuous. This also has nothing to do with the continuity of a representation...$GL(V)$ itself does not act continuously on $V$. I suppose you could restrict to $GL_c(V)$, the group of continuous invertible linear transformations. Commented Jul 16, 2015 at 5:14
• Sorry, that was what I meant. It seems to me that in texts about representation theory, it is always implicitely assumed that the operators $\Phi(g)$ are continuous. Commented Jul 16, 2015 at 5:38
• So my previous comment is wrong: one also consider non-continuous representations. But the map $\Phi$ always maps to the algebra of all linear invertible bounded endomorphisms of $V$. Commented Jul 16, 2015 at 5:40

The continuity/smoothness assumptions to require from infinite dimensional representations are a rather subtle issue. You can see that smoothness fails for very simple and natural examples. For example, take a compact Lie group $G$ and the Banach space $V:=C(G,\mathbb C)$ of complex valued continuous functions on $G$. The left regular representation of $G$ on $V$ defined by $(g\cdot f)(h):=f(g^{-1} h)$ for $f\in V$ and $g,h\in G$, is about the most natural representation that you can think of.
But for this representation, not even the map $g\mapsto g\cdot f$ (for fixed $f\in V$) is smooth. Indeed, if it were, you can compose it with the bounded linear (and hence smooth) evaluation map in the point $e\in G$. This gives $g\mapsto (g\cdot f)(e)=f(g^{-1})$, which is not smooth unless $f$ is smooth.
This illustrates the general phenomenon that $g\mapsto g\cdot v$ is smooth (as a map $G\to V$) for the elements of a dense linear subspace of $V$. This is called the space of smooth vectors, and the infinitesimal action of the Lie algebra is only defined on this subspace. (In practice, one uses an even smaller dense subspace, but that's a different issue.)
The correspondence between invariant subspaces for $G$ and $\mathfrak g$ in any case works only for closed subspaces, since derivatives of curves lying in a subspace have only vlaues in the closure of this subspace. You also have to pass to the closure for passing in the opposite direction. (So things are simple in the finite dimensional case only since there all subspaces are closed ...)
• I am not sure, but I think "Garding subspace" is synonymous for "space of smooth vectors". Even with the left regular representation, there is quite a bit of choice. One would oftenuse $L^2$ functions rather than continous ones. I have used $C(G,\mathbb C)$ for compact $G$ for the sake of argument, since it is clearly a Banach space. The smaller dense subspace I mentioned comes from requiring vectors in addition to be $K$-finite, i.e. contained in a finite dimensional subspace which is invariant under the maximal compact subgroup $K$ of $G$. Commented Jul 26, 2015 at 11:58