Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to create a simple implementation of the Bayes decision rule with minimum error criterion and I am running into a problem. Specifically, if I have a data set consisting of a number of feature vectors stored in rows, how can I generate a probability density function from this data?

Also, how can I do this if some of the data is discrete, some is continuous, and some is missing? For example, let us assume each feature vector, x, has three elements.

x = [ a, b, c]


  • a is categorical data and will be an element of the set {0, 1, 2, 3}
  • b is continous data and will be in the range [0,1]
  • c is also continous data in the range [0,1], but may be missing for some feature vectors

I want to be able to calculate the likelihood of a feature vector, x, based on the total data set or given that x is from a subset, w, of the total data set.

p(x) = ? and p(x|w) = ?

share|cite|improve this question
Have you gone about finding the distribution as a mixed distribution using just the frequency interpretation of probability? (From the obtained data) – Sudarsan Oct 5 '13 at 1:26
Sudarsan, thanks for the reply. The reason why I have not tried that is that my feature vectors have ~100 elements, many of which are continuous. For that reason, I suspect that none of the feature vectors will be identical. In that case, won't the probability of getting each feature vector just be equal to 1/N where N is the total number of feature vectors? – mblem22 Oct 5 '13 at 1:46
@MichaelBlemberg I have a similar question. If you find the answer please do post it. Thanks. – Sohaib I Oct 26 '13 at 7:08
@MichaelBlemberg You could use covariance as a measure in this case? – Sohaib I Oct 26 '13 at 7:11

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.