What is the difference between differentiability of a function and continuity of its derivative? I am sort of confused regarding differentiable functions, continuous derivatives, and continuous functions. And I just want to make sure I'm thinking about this correctly.
(1) If you have a function that's continuous everywhere, then this doesn't necessarily mean its derivative exists everywhere, correct? e.g., $$f(x) = |x|$$ has an undefined derivative at $x=0$
(2) So this above function, even though its continuous, does not have a continuous derivative? 
(3) Now say you have a derivative that's continuous everywhere, then this doesn't necessarily mean the underlying function is continuous everywhere, correct? For example, consider
$$
f(x) = \begin{cases}
1 - x \ \  \ \ \ x<0 \\
2 - x \ \ \ \ \ x \geq 0
\end{cases}
$$
So its derivative is -1 everywhere, hence continuous, but the function itself is not continuous? 
So what does a function with a continuous derivative say about the underlying function? 
 A: As you have shown the continuity of the derivative function doesn't guarantee that the function is itself continuous since it can be piecewise continuous.
WHat we can show is Continuous differentiability implies Lipschitz continuity.
A: A function may or may not be continuous.
If it is continuous, it may or may not be differentiable. $f(x) = |x|$ is a standard example of a function which is continuous, but not (everywhere) differentiable. However, any differentiable function is necessarily continuous.
If a function is differentiable, its derivative may or may not be continuous. This is a bit more subtle, and the standard example of a differentiable function with discontinuous derivative is a bit more complicated:
$$
f(x) = \cases{x^2\sin(1/x) & if $x\neq 0$\\
0 & if $x = 0$}
$$
It is differentiable everywhere, $f'(0) = 0$, but $f'(x)$ oscillates wildly between (a little less than) $-1$ and (a little more than) $1$ as $x$ comes closer and closer to $0$, so it isn't continuous.
A: Continuous but non-differentiable?

If you have a function that's continuous everywhere, then this doesn't necessarily mean its derivative exists everywhere, correct? e.g., $f(x)=|x|$ has an undefined derivative at $x=0$

...yes, but that's IMO not really a good way to talk about it. Saying “its derivative”, meaning the derivative at a single point, but then arguing that it doesn't exist is... ok, but if you come this angle then you should better say a derivative doesn't exist at $x=0$. I.e., there does not exist any affine function $g_0$ such that the difference between $f$ and $g_0$ vanishes with $\mathcal{O}(\Delta x^2)$ around $x=0$. Or simply, $f$ is not differentiable.
The reason we can talk about “the” derivative elsewhere is that (by design) if any derivative exists, then it is the only, unique one. But if none exists then, well, there can't be a unique one.

So this above function, even though its continuous, does not have a continuous derivative?

Here it's the other way around: now you're talking about the derivative as a function. This already presumes that you've checked differentiability everywhere in the domain. But as we've said there does not exist a local derivative at $x=0$, thus there also doesn't exist the derivative as a function and so any discussion about whether or not it's continuous are meaningless.
$f$ does, however, have generalised / weak derivatives, in an integral sense. For instance
$$
  g_\mathrm{l}(x) = \begin{cases}-1 & \text{for $x\leq 0$} \\ 1 & \text{else}\end{cases}
$$
fulfills
$$
  \int\limits_0^x\mathrm{d}\xi \: g_\mathrm{l}(\xi) = f(x).
$$
But $g_\mathrm{l}$ is not unique in this regard: the (Lebesgue) integral is never influenced by changes in a single point (or indeed in any countable set of points, generally in any null set). In particular, you could have also chosen
$$
  g_\mathrm{r}(x) = \begin{cases}-1 & \text{for $x< 0$} \\ 1 & \text{else}\end{cases}
$$
or
$$
  g_\mathrm{m}(x) = \begin{cases}-1 & \text{for $x< 0$} \\ 0 & \text{for $x=0$} \\ 1 & \text{else.}\end{cases}
$$
None of these are, of course, continuous. In that sense, “$f$ has discontinuous derivative” is kind of “physicist-correct”, but not really rigorous in the standard sense, which is set up in such a way that if any derivative exists, it is the unique derivative. For $f(x) = |x|$, all of $g_\mathrm{l}$, $g_\mathrm{m}$ and $g_\mathrm{r}$ can equally well claim to be derivatives of $f$, but all just weak derivatives. None is the derivative.
Discontinuous but differentiable?

For example, consider
  $f(x)=\begin{cases}1-x  &   x<0 \\ 2-x  &   x\geq0\end{cases}.$
  So its derivative is -1 everywhere, hence continuous, but the function itself is not continuous?

Noo, “its derivative is -1 everywhere” is wrong because it doesn't have a derivative. Can't be differentiable since it's not even continuous!
What you have implicitly done in your reasoning is separate the function in two domains
$$\begin{align}
  f_\mathrm{l} & \colon\quad ]-\infty,0[ \to \mathbb{R},& f_\mathrm{r} &\colon\quad ]0,\infty[ \to \mathbb{R}
  \\ f_\mathrm{l}&(x) = 1-x  & f_\mathrm{r}&(x) = 2-x.
\end{align}$$
Then both $f_\mathrm{l}$ and $f_\mathrm{r}$ are indeed differentiable, and for each the derivative is a constant function that gives $-1$ everywhere. But they are still given on different domains:
$$\begin{align}
  f_\mathrm{l}' & \colon\quad ]-\infty,0[ \to \mathbb{R},& f_\mathrm{r}' &\colon\quad ]0,\infty[ \to \mathbb{R}
  \\ f_\mathrm{l}'&(x) = -1  & f_\mathrm{r}'&(x) = -1.
\end{align}$$
These are not the same function. And this tells you nothing about differentiability of $f$ on its whole domain $\mathbb{R}$. In fact $f$ is not even weakly differentiable – for any weakly differentiable function on $\mathbb{R}$ there is a continuous function that differs from it in only a null set, but there's no way to smoothen $f$ to something continuous without changing its values on a whole open set around 0.
A: *

*Indeed.

*Indeed, the derivative does not even exist at $x=0$ as you argued in (1).

*Continuity is a necessary condition for differentiability (but it is not sufficient, as you have found in (1)). Your example is not differentiable at $0$, since it is discontinuous there. In particular, you can see this directly since
$$ \lim_{h\to0^+} \frac{f(h)-f(0)}{h} = \lim_{h\to0^+} \frac{(2-h)-2}{h} = -1, $$
whereas
$$\lim_{h\to0^-} \frac{f(h)-f(0)}{h} = \lim_{h\to0^-} \frac{(1-h)-2}{h} = \lim_{h\to0^-} \frac{-1-h}{h}$$
does not even exist.

A: The derivative is just another function, whether it's continuous doesn't affect the original function.
You have yourself given many examples on that.
Edit: in the further comments @gimusi has pointed we can show that Continuous differentiability implies Lipschitz continuity
A: Continuous everywhere, but differentiable nowhere: I present to you the Weierstrass function. It is defined as:
$$ W(x) = \sum_{n=0}^\infty a^n\cos(b^n\pi x).$$
You can find a visualization of $W$ here.
