What sets can we define continuity and differentiability on? I am an undergraduate Physics student (completing my first year shortly) who has had a (first) course on Calculus, and another on Linear Algebra.
When working with differential equations (in physical problems), I decided to look with mathematical rigour what I was dealing with. So I decided to start with the beginning. The following questions popped up.
Let $\Bbb{F=R}$ or $\Bbb{C}$.
Generalising Intervals
What is the concept parallel to an interval in $\Bbb{R}$, in $\Bbb{C}$? (Note: I’m not looking for contours, rather something which generalises the “rectanglular” and “circular” domains.)
Can the answer be open or closed subsets of $\Bbb{C}$? Can it be generalised to $\Bbb{F}^n$?
Generalising Continuity and Differentiability
Suppose we have a subset $S\subseteq \Bbb{F}$. What properties must $S$ have so that some form of continuity and differentiability can be defined on some subset $\mathcal{S} \subseteq S$?
(See this post (later in the post) and this.)
(I’d appreciate if your answer is generalisable to $\Bbb{F}^n$.)
Extending Normal Calculus
For such a “continuous and differentiable” $\mathcal{S} \subseteq S$, what theorems of normal calculus will apply?
Also, how will the integral, if needed, be redefined?
Finally, ODE’s
Now turning to ODE’s, I found a very satisfying definition of an ODE here. But unfortunately, it is restricted to intervals of $\Bbb{R}$. How can you generalise this definition to $\Bbb{C}$? (That’s why I was asking for generalisation of interval.)
Very importantly, can we have solutions to ODE’s not as “intervals”, but sets like $\mathcal{S}$ above?
 A: This is well-trodden ground in mathematical analysis. You should be able to find this information in nearly any text which covers multivariable calculus. See for example Chapter 3 of Apostol's Mathematical Analysis: Point Set Topology.
Point Set Topology is the field of mathematics that generalises notions of "interval" and "continuity", even beyond $\Bbb R^n$ and $\Bbb C^n$.
Differentiability has a number of generalisations, from gradients (in $\Bbb R^n$), which you will see a lot of in Physics, to the more abstract Fréchet derivative, which again works in more general spaces than $\Bbb R^n$.
Differentiability of complex functions is surprisingly different to the real case. A classic reference for this is Ahlfors' Complex Analysis. It covers a lot of ground and rigorously.
Not sure how much that will help in the direction of complex ODE's. But for the real case, when you add dimensions, you involve partial derivatives and end up with partial differential equations (PDE's). These are notoriously complicated to solve, but they can still be fruitfully studied. A search for "introduction to PDE's" or "first course in PDE's" will help. I don't want to recommend anything specifically, since I don't know the area.
I hope this gives you some ideas of where to look.
Update
Section 4 of Chapter 8 (Global Analytic Functions) of Ahlfors’ Complex Analysis is dedicated to linear complex differential equations. At least the material figures in the 3rd edition (1979).
