Typically, $e^x$ is defined as that series, and the fact that it converges it sufficient for us to say that the definition makes sense. Showing convergence is trivial - the ratio test establishes that ratio of common terms always goes to $0$, so it must converge. So, the only thing we can really do is to show that the series definition of $e^x$ does follow our intuition of how an exponential function should operate.
In particular, exponential functions satisfy the following identity:
$$e^ae^b=e^{a+b}$$
and this is the primary thing which makes them "exponential" - as it implies that $$e^n=\underbrace{e^1\cdot e^1 \cdot \ldots \cdot e^1\cdot e^1}_{n\text{ times}}.$$
If we wish to check that this property holds, we can expand the power series of both sides to get:
$$\left(\sum_{i=0}^{\infty}\frac{a^i}{i!}\right)\left(\sum_{i=0}^{\infty}\frac{b^i}{i!}\right)=\sum_{i=0}^{\infty}\frac{(a+b)^i}{i!}.$$
Now, we are allowed to do a lot of manipulations to these sums given that every sum is absolutely convergent. You can prove the assertions about sums I am making here, if you desire a simple exercise in analysis. Firstly, we can distribute the product of two sums into a double sum. In particular, on the left hand, we can write:
$$\sum_{i=0}^{\infty}\sum_{j=0}^{\infty}\frac{a^ib^j}{i!j!}=\sum_{i=0}^{\infty}\frac{(a+b)^i}{i!}$$
Which gives us each term $a^ib^j$ separately on the left. So, we then seek to isolate the same term on the right. In particular, notice that if we expand every $(a+b)^i$ by the binomial theorem, we get
$$\sum_{i=0}^{\infty}\sum_{j=0}^{\infty}\frac{a^ib^j}{i!j!}=\sum_{i=0}^{\infty}\sum_{k=0}^{i}{i \choose k}\frac{a^kb^{i-k}}{i!}$$
or, expanding ${i \choose k}=\frac{i!}{k!(i-k)!}$ gives:
$$\sum_{i=0}^{\infty}\sum_{j=0}^{\infty}\frac{a^ib^j}{i!j!}=\sum_{i=0}^{\infty}\sum_{k=0}^{i}\frac{a^kb^{i-k}}{k!(i-k)!}$$
but, since any sum over the same terms given here is absolutely convergent (you can use a sort of "two dimensional" ratio test to establish this), we can legally rewrite the left side by a change of variables with $j=i-k$ to get:
$$\sum_{i=0}^{\infty}\sum_{j=0}^{\infty}\frac{a^ib^j}{i!j!}=\sum_{i=0}^{\infty}\sum_{j=0}^{\infty}\frac{a^ib^{j}}{i! j!}$$
which, of course, holds and both sums converge absolutely.
The function defined by the series, therefore, does satisfy this exponential identity. We can also check that $e^1=e$, though that depends on how we want to define $e$ - but it should be hard in any case. Also $e^x$ is clearly increasing for positive $x$, since all the terms $\frac{x^i}{i!}$ are increasing on that domain. Moreover, we can prove it is increasing everywhere, since if $e^{-n}e^x=e^{x-n}$ is increasing when $x$ is positive (if we take $n$ as a constant), which, for suitably large $n$, proves that $e^{x-n}$ is increasing for all positive $x$ and ergo $e^x$ is positive for all $x$ greater than $-n$. I can't think of anything else you would demand from an exponential function - so clearly, it must be that the series definition of $e^x$ does make sense.
