Look up the concept of an unbounded operator. These are exclusive to the infinite dimensional setting.
The differences between finite dimensional linear algebra and analysis on infinite dimensional space are many and important. Basically, the entire game changes significantly. Of special importance to quantum mechanics, I would point out:
In the finite dimensional setting, all operators are bounded, and injectivity is equivalent to surjectivity. The spectrum of any operator is finite, and any point in the spectrum is an eigenvalue. Any normal operator has an orthonormal eigenbasis.
In the Hilbert space setting, boundedness is a non-trivial condition (in particular, common operators such as position and momentum operators are unbounded). Injectivity and surjectivity is no longer equivalent. The spectrum of a bounded operator can be any compact set, and no points in the spectrum have to be eigenvalues. The spectrum of an unbounded operator is non-trivial only if the operator is closed, and in that case the spectrum can be any closed set.
As an interesting aside, it is not unusual that powerful results in the Hilbert space setting are proven by a reduction to the finite dimensional setting. Loewner's Theorem on operator monotone functions is an example.