Im in my senior year of a math bachelors, taking a special topics course whose kinks and curriculum havent even been worked out yet - its more of a graduate course. Anyway, I am just now being introduced to this concept.
I believe the solution lies in the concept of the subgradient.
What is a subgradient? Basically, its the set of all possible gradients at a given point. For a differentiable function the subgradient contains a single value, the gradient that point. For a non-differentiable point, the subgradient contains every possible value, from the derivative at one side of the point to the derivative at the other, as an interval.
So if you look strictly at the function $|x|$, the derivative is $-1$ when $x<0$, it is $1$ when $x>0$. But at $x=0$ it is the set of all values in $[-1,1]$. Because we are sort of defining a derivative at $x=0$, we can get away with the chain rule.
Given that $f_1(x) = |x|^2$ we can differentiate this as $f_1'(x) = 2x\frac{x}{|x|}$, where we are representing the derivative of $|x|$ at non-zero values as $\frac{x}{|x|}$. But what about at $x=0$, that is the question.
Using the subgradient, let us represent the "derivative" at $x=0$ of $f_1$ as $\partial f_1(x)=2x\cdot[-1,1]$. I used the chain rule - which is allowable because I defined the derivative of the functions with the subgradient. Remember, we have continuity, just not standard differentiability, but we've defined a pseudo-derivative in its place. And in a way of thinking, the derivative becomes continuous too when you think of sliding across the interval.
Here, the interval $[-1,1]$ represents all possible gradients at $x=0$ of the $|x|$ function by itself. When we plug in $x=0$ - bare in mind the expression is only valid for $x=0$ anyway - then the subgradient reduces to a single value of $0$ because of the extra factor $x$. And $0$ is what we expect for a derivative of this function.
Similarly, if we try for $f_2(x)=|x^2|$ we get for non-zero values $f_2'(x) = \frac{x^2}{|x^2|}2x$, where $\frac{x^2}{|x^2|}$ is representative of the derivative at non-zero values, as before. But at $x=0$ the subgradient becomes the interval $[1,1]=1$ of possible instantaneous slopes. So in fact $\partial f_2(x) = 2x\cdot 1$. At $x=0$, which is the only point this expression is valid for anyway, the gradient reduces to $0$. As expected.
You can also find optimum points using subgradient theory. In order to have a maximum or a minimum, the subgradient must contain a $0$. Notice that for the function $|x|$, at $x=0$ we have $0\in [-1,1]=\partial |x|$, and so we have found an optimum. This would not have been possible with standard differential calculus, though its an obvious graphical solution.