I feel like this is not going to be a popular answer among statisticians, but this is how I think about the whole story. Here goes, hopefully it is of some help.
Let me explain the standard fuzz in a few lines: in the background, what statisticians have is a parametric family of probability distributions $\{\Bbb P_\theta : \theta \in \Theta\}$. You don't know the law of the random variables at hand; they can be anything depending on what the parameter $\theta$ is. Goal of a statistician is to guess what the distribution is from the experiments. Remember throughout that $\theta$ is unknown, varying in the parameter-space $\Theta$.
Let's say as a simplifying assumption that the range of the probability distribution is all of $\Bbb R$, and let's also assume these are absolutely continuous. If one does $n$ experiments, the results are the random variables $X_1, \cdots, X_n \sim \Bbb P_\theta$. The join vector $(X_1, \cdots, X_n)$ is something in the Euclidean space $\Bbb R^n$. This is the space of all possible results of $n$ experiments. A statistic is pretty much just a measurable function $T : \Bbb R^n \to \Bbb R$. I find it convenient to think of it as a partition of the space of results of $n$ experiments, $\Bbb R^n$, into a continuum of subsets $\{T^{-1}(r) : r \in \Bbb R\}$. A foliation in the geological sense if you wish (to a differential geometer, a foliation means something a bit more specific).
The varying probability distributions $\Bbb P_\theta$ can be thought of as a heat-distribution on $\Bbb R^n$. The law of $(X_1, \cdots, X_n)$ is $\Bbb P_\theta^{\otimes n}$, of course, so one can imagine that an evolution of a heat distribution on $\Bbb R^n$ with time $\theta$; at time $\theta = \theta_0$, some places are "hotter", aka there's more probability for the results of the $n$ experiments to land there, as opposed to other places which are "colder", which are sort of outlier regions for the result of the experiments. The heat diagram at time $\theta = \theta_0$ is the same as what simulation of the results of the $n$ experiments for the parameter $\theta = \theta_0$ would give.
On thing to note is that the $T$ associates to each leaf $T^{-1}(r)$ the number $r$. So our decomposition has moreover a value associated to each leaf; $T(X_1, \cdots, X_n)$ is the random variable which picks out a leaf at random according to the law of $\Bbb P_\theta$ (by picking a point at random from $\Bbb R^n$ according to $\Bbb P^{\otimes n}_\theta$, and then picking the unique leaf which passes through it, in case this is still unclear), and spits out the value of that leaf. To a mathematician, this is a probability distribution on a "space of leaves", $\Bbb R^n/T$ given by collapsing each leaf $T^{-1}(r)$ down to a point. Once again, $\theta$ is varying, so it is an evolution happening on the leaf space as well. The expected value $\Bbb E_\theta[g(T(X_1, \cdots, X_n))]$ is the expected value of the leaf, with respect to the probability-distribution on the leaf space induced from $\Bbb P_\theta$. So you can consider it to measure the "expected leaf".
Here's an immediate application of these intuitive ideas: What is a sufficient statistic? Recall the partition
$$\Bbb R^n = \bigsqcup_{r \in \Bbb R} T^{-1}(r)$$
Sufficiency demands that $T(X_1, \cdots, X_n)$ is independent of $\theta$; this is the same thing as saying that the heat evolution on $\Bbb R^n$ does not evolve on $T^{-1}(r)$'s for $r \in \Bbb R$. On the plaques or leaves of the "foliation", you cannot detect any change in the heat distribution as $\theta$ varies; indeed, the way heat evolves is if the leaves $\{T^{-1}(r) : r \in \Bbb R\}$ come closer, or move further. This "foliation" is the contour plot of the heat evolution! Another way to state it is $\Bbb P_\theta$ is completely determined by the induced distribution on the leaf space $\Bbb R^n/T$; there is no information "along the leaves", only "transverse to the leaves".
It's trickier to understand what completeness means. If $g : \Bbb R \to \Bbb R$ is another measurable function, then $g \circ T : \Bbb R^n \to \Bbb R$ is a new statistic, obtained from scrambling the values of the statistic $T$ by $g$ (this scrambling is not too drastic, in other words it's a measurable scrambling). The new decomposition is
$$\Bbb R^n = \bigsqcup_{r \in \Bbb R} T^{-1}(g^{-1}(r))$$
This is obtained from the old decomposition by simply collecting some of the leaves of the older foliation into a fat leaf of the new foliation. In other words, $T^{-1}(g^{-1}(r))$ is a union of the $T^{-1}(s)$, $s \in g^{-1}(r)$. Alternatively the leaf space $\Bbb R^n/g \circ T$ is a further quotient of $\Bbb R^n/T$.
Completeness is the same as saying that unless $g \circ T \equiv c$ is a constant a.e., i.e., the new scrambled foliation is the trivial one (if $T \equiv \mathrm{const}.$, what's the foliation corresponding to $T$ of $\Bbb R^n$?), one can find $\theta_0 \in \Theta$ such that expected leaf at $\theta = \theta_0$ has some value other than $c$. The expected leaf cannot have the same value throughout the heat-evolution; the value (i.e., the statistic $T$ or $g \circ T$ after scrambling) is supposed to capture something about the change of the heat-distribution, in expectation. That's all.
