Why do we need a function to be defined in a neighbourhood around $a$ so that it's differentiable at $a$? Compare it to the definition of continuity of a function $f$, if $x \in B(a,\delta)$, then $f(x) \in B(f(a),\varepsilon)$. This definition doesn't need $f$ to be defined around a neighbourhood around $a$, in fact $a$ can be an isolated point in the domain of $f$.
(I just checked), when we talk about the (single-var case) derivative of $f$ at $x_0$, we do not need $f$ to be defined in a neighbourhood around $x_0$.
$$
\lim_{x\to x_0} \frac{f(x)-f(x_0)}{x-x_0} = f'(x_0) \iff |x-x_0|< \delta \implies |\frac{f(x)-f(x_0)}{x-x_0} - f'(x_0)| < \varepsilon.
$$
We just need that if $B(x_0,\delta)$ contains points, then the inequality above holds.
However, in the $f:\mathbb{R}^p \to \mathbb{R}$ case, $f$ differentiable at $a \iff \exists l(x)$ linear function st. $f(x) = f(a) + l(x-a) + \varepsilon(x)\cdot|x-a|$, where $\varepsilon(x) = 0$ if $x \to a$. Why do we need $f$ to be defined in some neighbourhood here? The equivalent condition for differentiability of $f$ at $a$ is:
$$
\lim_{x\to a} \frac{f(x)-f(a)-l(x-a)}{|x-a|} = 0 \iff \forall x \in B(a,\delta), |\frac{f(x)-f(a)-l(x-a)}{|x-a|}| < \kappa.
$$
I don't understand why we need $f$ to be defined in some neighbourhood around $a$.
 A: I think part of your confusion is that continuity depends both on the function and the domain. For instance, the function
$$
f(x)=
\begin{cases}
0, x\in \mathbb{Q}\\
1, x\in \mathbb{R}\setminus\mathbb{Q}
\end{cases}
$$
isn't continuous on $\mathbb{R}$, but is continuous if you restrict to either of the rationals or irrationals - i.e. remove all of the discontinuities from your domain.
As for the derivative, a quick scan of my bookshelf (Baby Rudin, Bartle and Sherbert, Abbot) shows that it is common to require that $f$ is defined in an open ball containing $x$ for computing $f'(x)$ in the 1 dimensional case. Requiring that the function is defined on an open ball forces the object that you get out as the (total) derivative is a linear approximation in all directions from the point $x$. Allowing derivatives for functions not defined on an interval is basically just restricting the possible directions for your derivative to approximate in - with  the derivative for an isolated point being 0, since there are no "directions".
Also note that if you don't require definition on an open ball, then you can take derivatives at discontinuities, which will break the nice formulas we learn in calculus for computing derivatives of combinations of functions in the larger space.
