Continuous but non-differentiable?
If you have a function that's continuous everywhere, then this doesn't necessarily mean its derivative exists everywhere, correct? e.g., $f(x)=|x|$ has an undefined derivative at $x=0$
...yes, but that's IMO not really a good way to talk about it. Saying “its derivative”, meaning the derivative at a single point, but then arguing that it doesn't exist is... ok, but if you come this angle then you should better say a derivative doesn't exist at $x=0$. I.e., there does not exist any affine function $g_0$ such that the difference between $f$ and $g_0$ vanishes with $\mathcal{O}(\Delta x^2)$ around $x=0$. Or simply, $f$ is not differentiable.
The reason we can talk about “the” derivative elsewhere is that (by design) if any derivative exists, then it is the only, unique one. But if none exists then, well, there can't be a unique one.
So this above function, even though its continuous, does not have a continuous derivative?
Here it's the other way around: now you're talking about the derivative as a function. This already presumes that you've checked differentiability everywhere in the domain. But as we've said there does not exist a local derivative at $x=0$, thus there also doesn't exist the derivative as a function and so any discussion about whether or not it's continuous are meaningless.
$f$ does, however, have generalised / weak derivatives, in an integral sense. For instance
$$
g_\mathrm{l}(x) = \begin{cases}-1 & \text{for $x\leq 0$} \\ 1 & \text{else}\end{cases}
$$
fulfills
$$
\int\limits_0^x\mathrm{d}\xi \: g_\mathrm{l}(\xi) = f(x).
$$
But $g_\mathrm{l}$ is not unique in this regard: the (Lebesgue) integral is never influenced by changes in a single point (or indeed in any countable set of points, generally in any null set). In particular, you could have also chosen
$$
g_\mathrm{r}(x) = \begin{cases}-1 & \text{for $x< 0$} \\ 1 & \text{else}\end{cases}
$$
or
$$
g_\mathrm{m}(x) = \begin{cases}-1 & \text{for $x< 0$} \\ 0 & \text{for $x=0$} \\ 1 & \text{else.}\end{cases}
$$
None of these are, of course, continuous. In that sense, “$f$ has discontinuous derivative” is kind of “physicist-correct”, but not really rigorous in the standard sense, which is set up in such a way that if any derivative exists, it is the unique derivative. For $f(x) = |x|$, all of $g_\mathrm{l}$, $g_\mathrm{m}$ and $g_\mathrm{r}$ can equally well claim to be derivatives of $f$, but all just weak derivatives. None is the derivative.
Discontinuous but differentiable?
For example, consider
$f(x)=\begin{cases}1-x & x<0 \\ 2-x & x\geq0\end{cases}.$
So its derivative is -1 everywhere, hence continuous, but the function itself is not continuous?
Noo, “its derivative is -1 everywhere” is wrong because it doesn't have a derivative. Can't be differentiable since it's not even continuous!
What you have implicitly done in your reasoning is separate the function in two domains
$$\begin{align}
f_\mathrm{l} & \colon\quad ]-\infty,0[ \to \mathbb{R},& f_\mathrm{r} &\colon\quad ]0,\infty[ \to \mathbb{R}
\\ f_\mathrm{l}&(x) = 1-x & f_\mathrm{r}&(x) = 2-x.
\end{align}$$
Then both $f_\mathrm{l}$ and $f_\mathrm{r}$ are indeed differentiable, and for each the derivative is a constant function that gives $-1$ everywhere. But they are still given on different domains:
$$\begin{align}
f_\mathrm{l}' & \colon\quad ]-\infty,0[ \to \mathbb{R},& f_\mathrm{r}' &\colon\quad ]0,\infty[ \to \mathbb{R}
\\ f_\mathrm{l}'&(x) = -1 & f_\mathrm{r}'&(x) = -1.
\end{align}$$
These are not the same function. And this tells you nothing about differentiability of $f$ on its whole domain $\mathbb{R}$. In fact $f$ is not even weakly differentiable – for any weakly differentiable function on $\mathbb{R}$ there is a continuous function that differs from it in only a null set, but there's no way to smoothen $f$ to something continuous without changing its values on a whole open set around 0.