Assessing a distribution from multiple samples of its mean I face a random variable whose distribution I don't know.
Someone draws a sample of k observations from a population and tells me their average. He repeats the process m times.
I assume m is in order of magnitude of hundreds.
If 1 < k < 20, What can I tell about the population variance?
What about other lower moments?
If k=1, I can trivially draw the emplirical distribution. What is the closest analoug for 1 < k < 20?
 A: I think you mean to say ``someone draws a sample of $k$ observations from a population,'' and so on. Unfortunately, some textbooks and software packages
use 'sample' where the proper terminology is 'observation'. (Although I like Minitab for many purposes, its 'dialog boxes' consistently misuse the
word sample.) 
Note that $SD(\bar X) = \sigma/\sqrt{k}$ 
and $Var(\bar X) = \sigma^2/k,$ where $\sigma^2$ is the population variance.
Now suppose you treat the $m$ sample means $\bar X$ (each based on $k$
individual observations) as data, and find their sample variance, calling
it $S_{\bar X}^2.$ Then $S_{\bar X}^2$ estimates $\sigma^2/k,$ so that
$kS_{\bar X}^2$ is an estimate of $\sigma^2.$
Now you have the mean of the $m$ sample means, as an estimate of the population
mean $\mu$ and $kS_{\bar X}^2$ as an estimate of the population variance $\sigma^2.$ With similar approaches, you might be able to get (decreasingly
efficient) estimates of the population skewness and kurtosis. (I say 'decreasingly  efficient' because, as in the 2nd Comment of @Gregory Grant,
you lose information about the shape of population distribution when you take means of individual observations. The CLT is your ``enemy'' here.) 
Estimating $\mu$ and $\sigma^2$ is not
the same thing as knowing the shape of the population, but it may be a
start toward describing the population.
However, if you were able to see all $km$ of the individual observations,
you could make a histogram of them and get some idea of the shape of the
population density function. Better yet, you could make an empirical CDF 
curve (ECDF) of the $n = km$ observations. (That is better than a histogram,
because information is lost in sorting data into histogram bins.) Sort the data, an ECDF is a step function, starting at $0$ before the first observation, jumping
up by $1/n$ at each observation,  and reaching $1$ by the last observation. 
For moderately large $n$, the ECDF should be a reasonable approximation of the population CDF.
Numerical example. With $m = 1000$ and $k = 100,$ we take $n = mk$ observations from
the population distribution $Gamma(shape=3, rate=1/5),$ which has
mean $\mu = 15$ and variance $\sigma^2 = 75.$ Below we see that
the grand mean of all $n$ observations is 15.05 and that their variance
is 74.8, both good estimates of their corresponding population
parameters. (Simulation and computations in R statistical software.)
 m = 1000;  k = 100;  x = rgamma(m*k, 3, 1/5)
 mean(x);  var(x)
 ## 15.0481
 ## 74.79534

Now split these observations into $m = 1000$ samples, each with $k = 100$
observations. Each of the 1000 rows in the matrix DTA is a sample.
We find the means a of each sample. These are the $\bar X$s above.
Then mean(a) is the grand mean mentioned above, which is a good
estimate of $\mu$. And k*var(a) is $kS_{\bar X}^2$ above.Again
these are good estimates of $\mu$ and (alsmost as) good of $\sigma^2.$
(A little information was lost for estimating $\sigma^2$ when we
use sample means to estimate.)
DTA = matrix(x, nrow = m)
 a = rowMeans(DTA)
 mean(a);  k*var(a)
 ## 15.0481
 ## 76.28247

There are three graphs below. (1) a histogram of the 1000 $\bar X$s,
which starts to look a lot like normal because of the averaging (averaging obscures the true ehape of the population distribution); 
(2) a histogram of all $n = km$ individual observations, along with
the density curve of $Gamma(3, 1/5);$ and (3) an ECDF plot of the
$n$ observations (in black), with the CDF curve of $Gamma(3, 1/5)$
running almost exactly through it (in green). [Maybe right-click on
the figure and select 'new window' for aa bigger version and better detail.]

