As I read an introductory statistics book, a question struck me which am curious to find a good answer for.


The author teaches that statistical classification (understand this to mean organizing raw data into say classes -e.g by grouping the data into equal-sized categories, and assigning them frequencies, like what happens in preparation of a histogram) is a form of data compression. He further teaches that much as this compression approach aids in extraction and learning of the important information embedded in the data, he warns that certain approaches (e.g the use of the mode as a measure of central tendency) risk leaking away important information.

Of significance, he states that the mean (a measure of central tendency) and variance (a measure of variation / dispersion) are most important.


Now, I wonder, assuming one was to follow this line of thinking, and given a data sample, proceeded to statistically-compress it (in the sense indicated above).

  1. What statistical compression mechanism is sufficient to be able to recover without any loss whatsoever, the original data?
  2. And assuming such a lossless recovery is not possible, how can one reconstruct the most statistically-close (a probably approximate) approximation to the original data?


What compression information is sufficient to best reconstruct (or approximate) the original dataset:

  1. Only the mean and variance?
  2. Knowledge of the probability distribution of the mean and that of the variance?
  3. Or a smaller, but more representative sample of the original population?
  4. Or just the probability distribution of the data?
  5. Any other better mechanism?

Illustration / Application / Clarification:

Asume am given the following datasets (each line is a different dataset):

66, 66, 66, 67, 67, 67, 68, 69

52, 53, 61, 67, 71, 72, 78, 82

43, 44, 50, 54, 67, 90, 91, 97

Note that the mean for all three of the above samples is 67. Now, assume there was some statistical operation(s),$f$, such that applying it to any of the above samples would yield a (smaller) set of information,$s$, from which the original dataset could be reconstructed (preferably without any loss -- but a probably approximate reconstruction is equally good for me). The problem in this case would be to find such a function $f$ that can yield the small set of information $s$, from which (via some reverse-transformation, $f'$), one can obtain the original data:

$$f(Data) = s$$ such that $$f'(s) = Data$$

And $f$ is an application of statistical methods.

  • $\begingroup$ "What statistical compression mechanism is sufficient to be able to recover without any loss whatsoever, the original data?" There are lots of ways you can transform data without losing anything, but most of them aren't useful. What is the intended application? $\endgroup$ – Qiaochu Yuan Jan 4 '13 at 10:46
  • $\begingroup$ @QiaochuYuan, would it clarify my problem if i add: using only basic statistical tools e.g mean, variance and probability distributions? $\endgroup$ – nemesisfixx Jan 4 '13 at 11:22
  • $\begingroup$ No. I don't know what you mean by compressing data using basic statistical tools. In general I am suspicious of such questions unless you have an intended application in mind, which has the tendency of dramatically focusing questions. $\endgroup$ – Qiaochu Yuan Jan 4 '13 at 11:50
  • $\begingroup$ @QiaochuYuan, is the added application / clarification helpful? $\endgroup$ – nemesisfixx Jan 4 '13 at 12:10
  • 1
    $\begingroup$ You can look into principal component analysis. Here's a page that briefly explains how it can be used for compression: willamette.edu/~gorr/classes/cs449/Unsupervised/pca.html $\endgroup$ – Rahul Jan 4 '13 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.