Here are a few comments:
First off, I think it should be acknowledged that the OP's question is well-warranted, and it is the subject of an ongoing debate in a variety of circles (abstract-mathematical or otherwise). See e.g. https://stats.stackexchange.com/q/118/297795 or https://mathoverflow.net/q/1048/66883.
Next, the question seems to me to be somewhat multifaceted, in that it is not at all obvious to me that "having deeper significance" and "having intuitive interpretation" are positively correlated. Granted, both of these are vague terms to begin with. On that note I'll try to address both of them separately.
I believe it is reasonable (from the point of view of abstract mathematics) to interpret "having deeper significance" to be related to a question of the type "What property or list of properties of the (nonlinear) functional called the variance determines it among some larger class of functionals, and up to which relation?" (think the characterization of the Laplacian among all constant coefficient linear differential operators). It is straightforward that this question too is too vague and highly area-dependent (not only in how it can be answered but even in its formulation); below I'll list some examples.
As for an intuitive interpretation, here is one way to think of it. Let us take for granted (as it's done traditionally) that the variance of a random variable ought to be its covariance with itself:
$$\operatorname{var}(X) = \operatorname{cov}(X,X).$$
The traditional formula for covariance is:
$$\operatorname{cov}(X,Y) = \mathbb{E}((X-\mathbb{E}(X))(Y-\mathbb{E}(Y))) = \mathbb{E}(XY) - \mathbb{E}(X)\mathbb{E}(Y).$$
Reverse engineering the first expression leads to a Hilbert space interpretation as in the answer of J.G.. The caveat is the artifact of the introduced quotient space (What makes "$X\sim Y \iff Y=aX+b$ for some $a,b\in\mathbb{R}, a\neq0$" special, from a Hilbert space theory point of view?).
I'd like to propose reverse engineering the second expression, which leads to a quasi-homomorphism interpretation (in the sense of http://perso.ens-lyon.fr/ghys/articles/groupscircle.pdf, p. 349). Indeed, let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space and consider the space $L^0((\Omega,\mathcal{F},\mathbb{P});\mathbb{R})$ of real valued random variables defined on it. This space is a (topological) vector space; and pointwise multiplication makes it a (topological) commutative algebra. Further we have the (partially defined) expectation operator $\mathbb{E}$ determined by $\mathbb{P}$:
$$\mathbb{E}: L^0((\Omega,\mathcal{F},\mathbb{P});\mathbb{R})\rightsquigarrow \mathbb{R}, X\mapsto \int_{\Omega} X(\omega)\, d\mathbb{P}(\omega).$$
("$\rightsquigarrow$" means "partially defined": Notation for "function from a subset of $X$ into $Y$"?)
When $X$ and $Y$ have finite expectation, we have that $\mathbb{E}(X+Y) - [\mathbb{E}(X)+ \mathbb{E}(Y)] = 0$ (which is a convoluted way of saying that expectation is linear), but even when all constituents are finite it is not the case that $\mathbb{E}(XY) - [\mathbb{E}(X) \mathbb{E}(Y)] = 0$ always; the extent to which this fails is exactly covariance. Specializing to variance we get that $\operatorname{var}(X)$ tells to which extent $\mathbb{E}$ fails to be multiplicative when restricted to the algebra generated by $X$.
I should note that I find this to be intuitive from the point of view of probability theory, in the sense that multiplicative properties of $\mathbb{E}$ (or of $\mathbb{P}$) (i.e. some form of independence) are at the core of probability theory, and it's what distinguishes probability theory from abstract measure theory. (I believe I read a similar statement in one of Terry Tao's blogs, which influenced this interpretation, but I don't remember which one.)
Of course it would be disingenuous of me to pretend that there are no caveats with this reverse engineering too (beyond the fact that it is reverse engineering). Here are some that comes to mind:
I don't think it is straightforward that variance and covariance (or correlation) ought to be related (or ought to be related the way they are), as most answers seem to be assuming (Of course one could also question the relation between standard deviation and variance, but this questioning seems to me to be significantly more pedantic.). In functional analytical language, one could prefer to consider Banach space theory with no reference to Hilbert space theory. Case in point: the covariance of $X$ and $Y$ "intuitively" (from the statistical point of view) says how the change in $X$ and the change in $Y$ are related; how is the variance of $X$ not a priori $1$, "intuitively"? (This reasoning is not completely off; indeed it leads to the correlation coefficient). Observe that of course the traditional definition of covariance determines the traditional definition of variance and vice versa (by polarization); what I am pointing out here is different.
In regards to the (first instance of) "intuitively" of the previous bullet I'd like to point out the paper "Co-Relations and Their Measurement, Chiefly from Anthropometric Data" by Galton which the contemporary statistics community seems to take as the starting point of the concept of correlation (see e.g. Stigler's "Francis Galton's Account of the Invention of Correlation"). I found the Galton paper valuable beyond historical value, since he is trying to justify why the mathematical gadget he introduces quantifies the mathematically vague concept he is interested in. What is not emphasized is that what Galton calls correlation is actually median absolute deviance from median. I haven't looked deeper into the statistics literature, but there are obvious inquiries required by this. Still, dismissively one could say there is nothing special about any of this (assuming the import of a concept from biology/genetics into mathematics approximates some intuitional geodesic). For more on this line of thought the book Modeling, Measuring and Managing Risk by Pflug and Romisch seems to be valuable.
There is plenty of stuff that I brush under the rug by taking expectation to be a partially defined operator. In particular, the behavior of $\mathbb{E}$ under $k$-tuple products are not clear (to me at least) from its behavior under double products. Though this interpretation could lead to moments (like many others commented) and more sophisticated tools.
On that note, what ought to be the covariance of three random variables $X,Y,Z$? A Hilbert space interpretation would lead one to think of a $3\times 3$ covariance matrix with entries binary covariance (similar to "matrix coefficients"), and this seems to be common in statistics/probability. From a dynamical point of view (which happens to be my point of view) it seems $\mathbb{E}(XYZ) - [\mathbb{E}(X)\mathbb{E}(Y)\mathbb{E}(Z)]$ (similar to "multiple mixing") is more interesting. $\mathbb{E}((X-\mathbb{E}(X))(Y-\mathbb{E}(Y))(Z-\mathbb{E}(Z)))$ is also valid to consider of course.
Independence and vanishing covariance are not the same as is well known; I hope I am not being misleading when I say "some form of independence" above. Related to this is the paper by Renyi I mention below.
- Now let us discuss the deeper significance of variance when interpreted in the way I did above. More explicitly, the question is of the following form: let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space, and $\mathcal{C}\subseteq L^0((\Omega,\mathcal{F},\mathbb{P});\mathbb{R})$ be a collection random variables whose variance is defined. $\mathcal{C}$ is further assumed to be closed under whatever operations one might need to be syntactic. Let us take an arbitrary (possibly nonlinear) functional $\mathfrak{F}:\mathcal{C}\to \mathbb{R}$. What properties imposed on $\mathfrak{F}$ would guarantee the existence of a function $\mathfrak{f}:\mathbb{R}\to\mathbb{R}$ such that $\mathfrak{F} = \mathfrak{f}\circ \operatorname{var}$? Further, which properties of $\mathfrak{F}$ would guarantee which properties of $\mathfrak{f}$? The way I interpret this is that $\mathfrak{f}$ is some deterministic dependency of the anonymous functional $\mathfrak{F}$ on $\operatorname{var}$. As an example, for $\mathfrak{F}: X\mapsto \operatorname{var}(aX+b)$, $\mathfrak{f}: x\mapsto a^2x$ does the job uniquely (i.e. $\operatorname{var}(aX+b) = a^2\operatorname{var}(X)$).
In this framework the first paper I'd like to mention is "Why the variance?" by Kagan and Shepp. They consider functionals of the form
$$\mathfrak{D}_{\phi,a}: X\mapsto \mathbb{E} (\phi(X-a)),$$
where $\phi\in C^0(\mathbb{R},\mathbb{R})$ and $a\in\mathbb{R}$. Observe that $\mathfrak{D}_{|\cdot|^2,\mathbb{E}(X)}(X)=\operatorname{var}(X)$. For arbitrary $\phi$ (and $X$), $a\mapsto \mathfrak{D}_{\phi,a}(X)$ need not have a minimum; even when it does have a minimum said minimum need not be $a=\mathbb{E}(X)$. Using a terminology from a follow-up paper by Fainleib titled "On a characterization of measures of dispersion", let us define the $\phi$-base $\mathbb{B}_\phi(\mathbb{P})$ of $\mathbb{P}$ to be the following collection of random variables:
$$\mathbb{B}_\phi(\mathbb{P})=\left\{X\in L^0(\Omega;\mathbb{R})\left\vert \min_{a\in\mathbb{R}}\mathfrak{D}_{\phi,a}(X) = \mathfrak{D}_{\phi,0}(X)\right.\right\}.$$
(It is instructive to consider the cases $\phi = \operatorname{id}_\mathbb{R}$, $\phi = $ constant, $\phi = |\cdot|$, $\phi= |\cdot|^2$. The $\phi$-bases are: nothing, everything, random variables with vanishing median w/r/t $\mathbb{P}$ and random variables with vanishing mean w/r/t $\mathbb{P}$, respectively.)
Also observe that we could have been explicit with the dependency of $\mathfrak{D}_{\phi,a}$ on $\mathbb{P}$ from the get-go, if we work with honest to heavens measurable functions (and not their equivalence classes modulo negligible sets).
Theorem (Kagan-Shepp): If $\phi\in C^0(\mathbb{R},\mathbb{R})$ is such that all bounded random variables with vanishing mean w/r/t $\mathbb{P}$ are in $\mathbb{B}_\phi(\mathbb{P})$, then $\phi(x)= Ax^2+B$ for some $A\in\mathbb{R}_{\geq0}$ and $B\in \mathbb{R}$.
I should note that they in fact prove the result for vector-valued bounded random variables (with the obvious modification to the conclusion).
It is a straightforward algebraic manipulation to see that for $\phi:x\mapsto Ax^2+B$ with $A>0$,
$$\mathfrak{D}_{|\cdot|^2,a}(X) = A\,(a- \mathbb{E}(X))^2 + \left[\mathfrak{D}_{|\cdot|^2,0}(X)- A\,(\mathbb{E}(X))^2 \right].$$
In particular the minimum of $a\mapsto \mathfrak{D}_{\phi,a}(X)$ is $\mathfrak{D}_{|\cdot|^2,0}(X)- A\,(\mathbb{E}(X))^2$, said minimum is attained exactly when $a=\mathbb{E}(X)$, and finally the $\phi$-base $\mathbb{B}_\phi(\mathbb{P})$ of $\mathbb{P}$ is exactly the collection of random variables with zero mean w/r/t $\mathbb{P}$ (i.e. the kernel of $\mathbb{E}$).
This shows that, as the authors declare, "the second moment is essentially the only characteristic defined for all bounded random variables $X$ that is minimized when taken around $a=\mathbb{E}(X)$", and the minimum value is exactly $\operatorname{var}(X)$.
About twenty years after the Kagan-Shepp paper a similar result, but with an emphasis on the additivity property of variance for independent random variables, is obtained by Poschadel in the paper "On a characterization of variance and covariance".
An alternative framework is used by Mattner in the paper "What are cumulants?". His emphasis is on the semigroup structure of the space $\operatorname{Prob}(\mathbb{R},\mathcal{B})$ of Borel probability measures on the real line given by convolution. Using the $k$-th derivative of (some branch of) logarithm applied to the characteristic function ( = Fourier transform) of a probability measure $\mathbb{P}$ he defines the $k$-th cumulant of $\mathbb{P}$, and the $2$nd cumulant of $\mathbb{P}$ turns out to be exactly the variance of $\mathbb{P}$. Here it is proved, among other things, that (modulo some details I don't want to get into here) the only continuous additive maps on the convolution semigroup of probabilities (actually the subsemigroup of probabilities with "finite moments of all orders") are constant multiples of variance. Observe that this fits into the more general framework I mentioned above, if we impose the condition on $\mathfrak{F}$ that it be "version-independent", i.e. if $X$ and $Y$ have the same law ( i.e. if $X_\ast(\mathbb{P})=Y_\ast(\mathbb{P})$, where the lowerscript $\ast$ signifies the pushforward), then $\mathfrak{F}(X)=\mathfrak{F}(Y)$ (See the book of Pflug and Romisch I mentioned above for more on version-independent "deviation-type" functionals, from the statistical point of view).
Finally let me mention the paper "On Measures of Dependence" by Renyi. He focuses on alternatives to the correlation coefficient, and he surveys a variety of gadgets according to whether or not each one of these gadgets satisfies the postulates he puts forth for a reasonable measure of dependence ought to satisfy. The traditional correlation coefficient (or its absolute value) fails to satisfy all said postulates, but the so-called maximal correlation, introduced by Gebelein, satisfies all.
