Relevance of smoothness. What is the real relevance of a function being smooth, other than this being necessary for analyticity. What is the real problem if the 4323rd derivative has a discontinuity?
 A: In practice, that is of little or no consequence. However, smoothness simplifies some proofs, and it also simplifies some definitions: For example, the characterization of a tangent vector at a point $p$ of a manifold $M$ being a linear functional $X$ on $C^\infty(M)$ satisfying $X(fg)=X(f)g(p)+f(p)X(g)$ is handy sometimes, but I don't think its analogue for $C^n(M)$ holds. However, you can do $C^n$ differential geometry with a more direct definition of tangent vectors.
Edit in response to a comment below: Let us work through the above in the case $M=\mathbb{R}^d$, letting $p$ be the origin. We note that for any $f\in C^\infty(\mathbb{R}^d)$
$$\begin{aligned}
  f(x)-f(0)&=\int_0^1 \frac{d}{dt} f(tx)\,dt=\sum_{j=1}^d x^j f_j(x),
  \quad\text{where}\\
 f_j(x)&=\int_0^1\partial_jf(tx)\,dt,
  \end{aligned}
$$
where $\partial_jf$ is the partial derivative of $f$ with respect to $x^j$, and I follow the common convention of writing $x^j$ for the $j$th component of $x$.
Now it is easy to see that $X(g)=0$ for a constant $g$. Also $f_j(0)=\partial_jf(0)$, so applying $X$ to the above we get
$$ X(f)=\sum_{j=1}^d X^j\partial_jf(0), \qquad\text{where $X^j=X(x^j)$.}$$
In brief, $$X=\sum_{j=1}^d X^j\partial_j.$$
In particular the space of linear operators $X$ satisfying $X(fg)=X(f)g(0)+f(0)X(g)$ is $d$-dimensional, and can be identified with the tangent space at $0$.
All this breaks down if I replace $C^\infty$ by $C^k$ for $k<\infty$! For one thing, if $f\in C^k$ then we cannot say $f_j\in C^k$ anymore, so the whole proof breaks down. Further, the space of “tangent vectors” $X$ becomes infinite dimensional! See [1]. (I learned this from Warner's Foundations of Differential Manifolds and Lie Groups.)
[1] W. F. Newns and A. G. Walker, Tangent planes to a differentiable manifold, J. London Math. Soc. (1956) 31, 400–407 (direct link to a pdf, perhaps behind a paywall).
A: The real advantage of smoothness is that assuming that a function is smooth frees you from keeping track of exactly how many of its higher derivatives you need to exist for your computations to be valid. It's just easier that way, and mathematicians are lazy.
It also helps that (in the real case) there are enough smooth functions that this assumption usually doesn't throw away anything you'll regret losing later. You can often get rid of the smoothness assumption by some (often implied) approximation argument at the end of the day after you've done the real work in a nice hypothetical world where everything is smooth.
In contrast, for example, assuming that everything is analytic is a lot more restrictive, and entails many consequences that are not even approximately true about less nice cases.
A: This is a great question, one I was thinking about when answering a question about finding the "knees" of a sigmoid, and there, I was having the OP take the third derivative of the curve to do so.  I was wondering how one could find a natural curve in which some high-order derivative was discontinuous, and how one could tell from the original curve.
Discontinuous derivatives play a role in the behavior of Fourier transforms.  Consider $f(x) = x^2 e^{-x} \theta(x)$, where $\theta(x) = 0 \, \forall \, x<0$ and is 1 otherwise.  Here is a plot:
 
Near $x=0$, you'd have a hard time telling that this function isn't smooth.  But, as it turns out, Fourier transforms of functions can pick these things out because FT's of such functions have algebraic behavior at $\infty$ rather than exponential behavior.  For example, the FT of $f(x)$ is
$$\hat{f}(k) = -\frac{i \sqrt{\frac{2}{\pi }}}{(k+i)^3} = O\left ( \frac{1}{k^3} \right )$$
as $k \rightarrow \infty$.  In general, when there is a discontinuity in the $m$th derivative of $f(x)$, the corresponding FT $\hat{f}(k) = O(1/k^{m+1})$.  So, in your hypothetical function with the discontinuity in the $4323$rd derivative, you could verify by taking the FT, plotting its absolute value on a log-log plot, and verifying that the slope of the line for large $k$ is indeed $-4324$.
