Of course, I assume the manifold is connected. I feel like this is probably true but I have no idea how to prove it. Any hints?
Tim's answer is spot on, but I wanted to mention that the answer switches if you assume the Lie group is compact.
Suppose $G$ is a compact Lie group acting on a connected manifold $M$. If $G$ fixes a non-empty open set pointwise, then the $G$ action on $M$ is trivial.
Proof: Pick any background Riemannian metric on $M$. By averaging this action over the $G$ action, we obtain a $G$-invariant Riemannian metric. In particular, we may assume $G$ acts isometrically.
Now, let $U$ be a non-empty open subset of $M$ on which $G$ acts trivially and let $p\in U$. Then, for any $g\in G$, we of course have $g\ast p = p$, but more is true: $d_p g:T_p M\rightarrow T_p M$ is the identity function. To see this, pick $v\in T_p M$ and let $\gamma$ be a curve with image entirely in $U$ for which $\gamma'(0) = v$. Then $g \gamma(t) = \gamma(t)$ and differentiation both sides at $t = 0$ gives $d_p g (v) = v$.
But an isometry of a connected Riemannian manifold is determined by its action at a single point. Since multiplication by $g$ and the identity map do the same thing at $p$, $g = Id$. In particular, $G$ acts trivially.