Indefinite Sum concept is the answer to your question.
In fact, if a function $f(x)$ is the (forward) difference of a function $F(x)$
$$
f(x) = \Delta _x F(x) = F(x + 1) - F(x)
$$
then we say that $F(x)$ is the "antidifference" (or "indefinite sum") of $f(x)$
$$
F(x) = \Delta _x ^{\left( { - 1} \right)} f(x) = \sum\nolimits_x {f(x)}
$$
If $ f(x)$ and $F(x)$ are defined over a real, or complex, domain for $x$ then we will have for example
$$
\sum\limits_{k = 0}^n {f(x + k)} = \sum\limits_{k = 0}^n {\left( {F(x + 1) - F(x)} \right)} = F(x + n + 1) - F(x)
$$
We write the above with a different symbol for the sum as
$$
\sum\nolimits_{k = 0}^{\,n} {f(x + k)} = \sum\limits_{k = 0}^{n - 1} {f(x + k)} = F(x + n) - F(x)
$$
which can also be written as
$$
\sum\nolimits_{k = x}^{\,x + n} {f(k)} = F(x + n) - F(x)
$$
The extension to
$$
\sum\nolimits_{k = a}^{\,b} {f(k)} = F(b) - F(a)
$$
for any real (or complex) $a,b$ inside the domain of definition of $f, F$ is quite natural.
Coming to the harmonic numbers, it is well known that the functional equation of the digamma function is
$$
{1 \over x} = \Delta _x \,\psi (x)
$$
and it is therefore "natural" to define
$$ \bbox[lightyellow] {
H_r = \sum\limits_{k = 1}^r {{1 \over k}} = \sum\nolimits_{k = 1}^{\,1 + r} {{1 \over k}} = \psi (1 + r) - \psi (1) = \psi (1 + r) + \gamma
} \tag{1}$$
which substantiate M. Janisch's comment
Answering to W. Hintze's comment, please consider how the "indefinite sum" parallels the "indefinite integral - antiderivative" concept.
Similarly to
$$
f(x) = {d \over {dx}}F(x)\quad \Leftrightarrow \quad F(x) = \int {f(x)dx} + c
$$
we have that
$$
f(x) = \Delta \,F(x)\quad \Leftrightarrow \quad F(x) = \sum\nolimits_x {f(x)} + \pi \left( x \right)
$$
where now the family of antidifference functions differ by any function $\pi(x)$ , and not by a constant, which is periodic
with period (or one of the periods) equal to $1$, as you rightly noticed.
So by "natural extension" I meant to say:
- an extension from integers to reals (and complex) field under the "indefinite sum" concept,
which provides a function $\mathbb C \to \mathbb C$ which fully interpolates $f(n)$;
- in the antidifference family to select the "simpliest / smoothiest" function, same as
the Gamma function is selected among the functions satisfying $F(z+1)=zF(z)$ as the
only one which is logarithmically convex,
or which has the simplest Weierstrass representation, etc.
From the comments I could catch some skepticism as if my answer could just be an "extravagant" personal
idea of mine.
That's absolutely not so, the definition in (1) is actually standardly accepted: refer to
Wolfram Function Site,
and in particular to this section or to the
Wikipedia article.
I am just trying to enlight how we can assign a meaning to sums with bounds which are not integral and thus saying that
$$ \bbox[lightyellow] {
H_{\,1 + \sqrt {5/2} } = \sum\limits_{k = 1}^{\,1 + \sqrt {5/2} } {{1 \over k}} = \sum\nolimits_{k = 1}^{\,\,2 + \sqrt {5/2} } {{1 \over k}} = \psi (\,\,2 + \sqrt {5/2} ) - \psi (1) = 1.7068 \ldots
} \tag{2}$$