As I read an introductory statistics book, a question struck me which am curious to find a good answer for.
The author teaches that statistical classification (understand this to mean organizing raw data into say classes -e.g by grouping the data into equal-sized categories, and assigning them frequencies, like what happens in preparation of a histogram) is a form of data compression. He further teaches that much as this compression approach aids in extraction and learning of the important information embedded in the data, he warns that certain approaches (e.g the use of the mode as a measure of central tendency) risk leaking away important information.
Of significance, he states that the mean (a measure of central tendency) and variance (a measure of variation / dispersion) are most important.
Now, I wonder, assuming one was to follow this line of thinking, and given a data sample, proceeded to statistically-compress it (in the sense indicated above).
- What statistical compression mechanism is sufficient to be able to recover without any loss whatsoever, the original data?
- And assuming such a lossless recovery is not possible, how can one reconstruct the most statistically-close (a probably approximate) approximation to the original data?
What compression information is sufficient to best reconstruct (or approximate) the original dataset:
- Only the mean and variance?
- Knowledge of the probability distribution of the mean and that of the variance?
- Or a smaller, but more representative sample of the original population?
- Or just the probability distribution of the data?
- Any other better mechanism?
Illustration / Application / Clarification:
Asume am given the following datasets (each line is a different dataset):
66, 66, 66, 67, 67, 67, 68, 69
52, 53, 61, 67, 71, 72, 78, 82
43, 44, 50, 54, 67, 90, 91, 97
Note that the mean for all three of the above samples is 67. Now, assume there was some statistical operation(s),$f$, such that applying it to any of the above samples would yield a (smaller) set of information,$s$, from which the original dataset could be reconstructed (preferably without any loss -- but a probably approximate reconstruction is equally good for me). The problem in this case would be to find such a function $f$ that can yield the small set of information $s$, from which (via some reverse-transformation, $f'$), one can obtain the original data:
$$f(Data) = s$$ such that $$f'(s) = Data$$
And $f$ is an application of statistical methods.