Can a function have a derivative where it has no value? Consider the following function: $$f(x) = x^2 \mid x \in \mathbb{R}, \ x \ne 0$$
The derivative at $x=0$ seems to want to be zero, in the same way that $\lim \limits_{x \to 0} f(x) = 0$ 
However, when I look at the definition of the derivative, this doesn't seem to work:
$$f'(x) =  \lim \limits_{\Delta x \to 0} \frac{f(x+\Delta x )-f(x)}{\Delta x}.$$ The function isn't defined at $f(x)$, so $f'(x)$ is also undefined. Would it make any sense to replace the $f(x)$ in the definition with $\lim \limits_{x \to 0} f(x) = 0$? Then, I suppose we'd have $\lim \limits_{x \to 0} f'(x) = 0$? 
Would it be permissible? Would there be any point?
 A: As has already been said, and as you have found yourself, the ordinary derivative is not defined. However, the symmetric derivative is defined in your case:
$$f'_{\text{symmetric}}(x_0) := \lim_{h\to 0} \frac{f(x_0+h)-f(x_0-h)}{2h}$$
A more general derivative is also defined (I'll call it excluding; I don't know if it has a name):
$$f'_{\text{excluding}}(x_0) := \lim_{h,k\to 0\\h,k \neq 0} \frac{f(x_0+h)-f(x_0+k)}{h-k} = \lim_{x_1,x_2\to x_0\\x_1,x_2 \neq x_0} \frac{f(x_1)-f(x_2)}{x_1-x_2}$$
A: If a function $f$ is not defined at a point $x_0$ we can't evaluate the derivative at $x_0$ because in the definition of the derivative at $x_0$ we need $f(x_0)$. 
However if $f$ is continuous in $0<|x-x_0|<r$ with $r>0$ and the limit $\lim_{x\to x_0}f(x)=L$ exists we can extend $f$ to a continuous function in $|x-x_0|<r$ by letting $f(x_0):=L$. 
Moreover if $f$ is also differentiable in $0<|x-x_0|<r$ and the limit $\lim_{x\to x_0}f'(x)=a$ then the extended function is also differentiable at $x_0$ and its derivative at $x_0$ is equal to $a$.
See also Prove that $f'(a)=\lim_{x\rightarrow a}f'(x)$.
A: No.
You can certainly extend $f$ to a function $g$ as you indicate, but then you are really computing $g'(0)$, not $f'(0)$.
In that case, $f'$ agrees with $g'$ on their common domain, which excludes $x=0$.
Remember, if a function is differentiable at $x_0$, then it is continuous at $x_0$. In particular, it is defined at $x_0$.
A: You function does have a derivative, just not a "classical" or "strong" one. Instead your function has what's called a "weak derivative", which is technically not a single function but rather a class of functions for technical reasons from measure theory about zero-measure sets, but this class usually (in practical problems) has a single continuous or piecewise continuous representative. In your case, the function $g(x) = 2x$ is the continuous representative of the class of weak derivatives of your function $f$, namely because for any "test function" $\phi$ (i.e. a smooth and compactly supported function) on $\mathbb{R}$ we see that the relation of integration by parts holds:
$$\int_\mathbb{R} f(x) * \phi'(x) dx = \int_\mathbb{R} x^2 * \phi'(x) dx = - \int_\mathbb{R} 2x * \phi(x) dx = \int_\mathbb{R} g(x) * \phi(x) dx$$
