Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am doing a course in Machine Learning, and I am having some trouble getting an intuitive understanding of maximum likelihood classifiers. What I am trying to do is to perform Principal Component Analysis on the Iris Flower Data Set, and then classify the points into the three classes, i.e. Setosa, Versicolor, Virginica.

So I take some points from each class of Iris to be used as training data, and transform it onto the principal component axes. Next I want to model each class as a multivariate Gaussian Distribution, estimating the parameters of each class from my training data. However, this is where I get stuck. I will try and explain this graphically using some (made-up) sample data:

Class 1       Class 2       Class 3
1 2 1 4       4 2 3 1       6 8 1 2
4 7 8 1       2 3 5 3       7 7 5 4
1 2 4 5       1 5 4 2       1 6 8 3

If this is the data for three different classes, with four observations and three features each, how would I go about determining the mean and covariance matrix for each class?

My original thoughts were that the mean of class 1 would be [mean(1,2,1,4), mean(4,7,8,1), mean(1,2,4,5)], but then what about the covariance?

I hope that I have explained this well enough, any help is greatly appreciated!

Thanks

Casper

PS. Also, this is my first time posting on the Mathematics part of StackExchange, so feel free to point out any etiquette-rules that I might have inadvertently violated.

share|improve this question

1 Answer 1

up vote 0 down vote accepted

In the case of class 1, for example, I think you can do

mu = mean(Data,2);
Sigma = cov(Data');
P = gausspdf(Data, mu, Sigma);

in MATLAB, where gausspdf.m is some Gaussian pdf building function.

If you want to build Gaussian mixtures, I think you can do

num = 4 % number of Gaussians in one mixture

[index center] = kmeans(Data',num);
for i = 1:1:num
  count = length(find(index==i));
  sigm(:,:,i) = cov(Data(:,index)');       
end

weight = count/sum(count);
mu = center';
sigma = sigm;
share|improve this answer
    
Thanks, I actually managed to get this working but forgot to post the answer here. Your answer agrees with mine though, so it seems I was correct. Thanks! –  casper Jul 9 '12 at 8:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.