I think this is only going part-way to what you want, because I still want to use a ``nice domain" first, but I'd like to make a case for the fact that using a convex domain in this way is not an artificial device.
Step 1: (this is the bit you're not going to like) We use Cauchy's theorem for disks to show a special case of Cauchy's integral formula: that is, we show that for $z \in B(a,r)$ the open disk of radius $r$ centred at $a$ we have $f(z) = \frac{1}{2\pi i} \int_{\gamma_{a,r}} \frac{f(w)}{w-z}dw$, where $\gamma_{a,r} = a+r\exp(2\pi i.t)$ is the closed path given by traversing the boundary of $D(a,r)$ anticlockwise.
This follows easily from Cauchy's theorem: by translating and scaling one can assume that $a =0$ and $z=s>0$ is a positive real number, so that $s\in (0,1)$. If we write $\gamma_{[z_0,z_1]}(t) = z_0+t(z_1-z_0)$ for the line segment path from $z_0$ to $z_1$, and $u_{a,r}(t) = a+ r\exp(\pi i.t)$, $l_{a,r}(t) = a-r\exp(\pi i.t)$ for the semicircular paths from $a+r$ to $a-r$ and $a-r$ to $a+r$ respectively, then if we fix $r<(1-s)/2$ and, for a path $\gamma\colon [0,1]\to \mathbb C$, write $\gamma^-$ for the path $\gamma^-(t) = \gamma(1-t)$, define
$$
\begin{split}
\Gamma_+ &= u_{0,1}*\gamma_{[-1,r-s]}*u_{s,r}^-*\gamma_{[s+r,1]} \\
\Gamma_- &= \gamma_{[s+r,1]}^-*l_{s,r}^-*\gamma_{[-1,s-r]}^-*l_{0,1}
\end{split}
$$
with $*$ denoting concatenation of paths.
Thus $\Gamma_{\pm}$ are "indented" versions of the closed paths obtained by traversing the interval $[-1,1]$ and one of semicircles joining $-1$ and $1$,
with semicircular ``indentations" of radius $r$ centred at $s$ ensuring that they avoid $s$. As a result, each of the paths $\Gamma_{\pm}$ are closed contours lying in a simply-connected domain in which the function $g(z) = \frac{f(z)}{z-s}$ is holomorphic, and so the integral over each is zero.
$\Gamma_{\pm}$" />
Now since the line segment paths cancel in the sum $\Gamma_+ + \Gamma_-$ and the semicircular paths combine to give $\gamma_{0,1} + \gamma_{s,r}^-$, we deduce that the integral $\int_{\gamma_{0,1}} \frac{f(z)}{z-s}dz = \int_{\gamma_{s,r}}\frac{f(z)}{z-s}dz$. Since it is easy to see just using the continuity of $f$ at $s$ that $\lim_{r\to 0} \int_{\gamma_{s,r}} \frac{f(z)}{z-s}dz = 2\pi i.f(s)$, it follows that $\frac{1}{2\pi i}\int_{\gamma_{0,1}} \frac{f(z)}{z-s} dz = f(s)$.
Step 2: Once you have this simple form of Cauchy's Integral Formula, you can prove all the good "local" properties of holomorphic functions, in particular it is easy to see that holomorphic functions are analytic, that is, if $f$ is holomorphic on an open set $U$ and $a \in U$, $f$ is equal to its Taylor series in any ball $B(a,r)$ strictly contained in $U$. Using ``nice domains" to prove local properties of holomorphic functions seems reasonable to me, because $\mathbb C$ (indeed any vector space) is locally convex, or "locally nice" so to speak.
Step 3: So now, finally, for what hopefully counts as the "good bit": if you know that holomorphic functions are analytic, you can give a proof of the "holomorphic except at one point implies holomorphic" statement which is (to my mind) much more satisfying than the method you outlined, because it feels more conceptual:
To clarify what we want to do, recall that if $f$ is holomorphic on a domain $D$ and $B(z_0,r)\backslash \{z_0\} \subseteq D$ and $f$ is bounded on $B(z_0,r)\backslash\{z_0\}$, then $f$ extends to a holomorphic function on all of $B(z_0,r)$ (so in particular, if $f(z)$ is continuous at $z_0$ then it was in fact already holomorphic there). The reason this is easy to believe when you know holomorphic implies analytic, is that it is easy to "squash" the singularity at $z_0$: Since $f$ is bounded near $z_0$, the function $(z-z_0).f(z)$ vanishes as $z\to z_0$. Thus if we set $h(z) = (z-z_0)^2f(z)$ for $z \neq z_0$ and $h(z_0)=0$ we have
$$
\lim_{z\to z_0} \frac{h(z)-h(z_0)}{z-z_0} = \lim_{z\to z_0} (z-z_0).f(z) \to 0, \text{ as } z\to z_0,
$$
so that $h'(z_0)=0$. Clearly $h(z)$ is holomorphic everywhere $f$ is, and hence it follows that $h(z)$ is holomorphic in $B(z_0,r)$. But then we know that, near $z_0$ (say on $B(z_0,r/2)$) the function $h(z)$ is given by a power series, that is $h(z) = \sum_{n\geq 0} a_n(z-z_0)^n$, and moreover because $h(z)=h'(z)=0$, standard properties of power series show that $a_0=a_1=0$, and so $h(z) = (z-z_0)^2.\sum_{n\geq 0} a_{n+2}(z-z_0)^n$. But then $f(z) = \sum_{n\geq 0} a_{n+2}(z-z_0)^n$ is analytic and hence holomorphic on $B(z_0,r/2)$, so that in particular if we set $f(z_0) = a_2$, then $f$ is holomorphic at $z_0$.
Step 4: Once we have the removable singularity result, we can easily obtain the integral formula from Cauchy's theorem using the secants of $f$: if $f(z)$ is holomorphic in a simply-connected domain $\Omega$, and $w\in \Omega$, the function $g(z) = \frac{f(z)-f(w)}{z-w}$ is clearly holomorphic in $\Omega\backslash\{w\}$, and as $z \to w$ clearly $g(z) \to f'(w)$, hence it is bounded. By the above, it follows that $g(z)$ is holomorphic on all of $\Omega$, whence $\int_{\gamma} g(z)dz = 0$ by Cauchy's theorem. But then it follows that
$$
\int_{\gamma} \frac{f(z)}{z-w} - \frac{f(w)}{z-w} dz =0,
$$
so that
$$
\frac{1}{2\pi i}\int_{\gamma} \frac{f(z)}{z-w}dz = \frac{f(w)}{2\pi i}\int_{\gamma} \frac{1}{z-w} dz = n(\gamma,w).f(w)
$$
where $\frac{1}{2\pi i} \int_{\gamma} \frac{dz}{z-w} = n(\gamma,w)$ is the winding number by definition for some people, but is readily identified with it however it is defined.
In conclusion, if you can put up with disks for a bit then this approach removes the need to consider any artificial points in the domain of $f$.