# Defining Classes for Maximum Likelihood Classifier

I am doing a course in Machine Learning, and I am having some trouble getting an intuitive understanding of maximum likelihood classifiers. What I am trying to do is to perform Principal Component Analysis on the Iris Flower Data Set, and then classify the points into the three classes, i.e. Setosa, Versicolor, Virginica.

So I take some points from each class of Iris to be used as training data, and transform it onto the principal component axes. Next I want to model each class as a multivariate Gaussian Distribution, estimating the parameters of each class from my training data. However, this is where I get stuck. I will try and explain this graphically using some (made-up) sample data:

Class 1       Class 2       Class 3
1 2 1 4       4 2 3 1       6 8 1 2
4 7 8 1       2 3 5 3       7 7 5 4
1 2 4 5       1 5 4 2       1 6 8 3


If this is the data for three different classes, with four observations and three features each, how would I go about determining the mean and covariance matrix for each class?

My original thoughts were that the mean of class 1 would be [mean(1,2,1,4), mean(4,7,8,1), mean(1,2,4,5)], but then what about the covariance?

I hope that I have explained this well enough, any help is greatly appreciated!

Thanks

Casper

PS. Also, this is my first time posting on the Mathematics part of StackExchange, so feel free to point out any etiquette-rules that I might have inadvertently violated.

-

In the case of class 1, for example, I think you can do

mu = mean(Data,2);
Sigma = cov(Data');
P = gausspdf(Data, mu, Sigma);


in MATLAB, where gausspdf.m is some Gaussian pdf building function.

If you want to build Gaussian mixtures, I think you can do

num = 4 % number of Gaussians in one mixture

[index center] = kmeans(Data',num);
for i = 1:1:num
count = length(find(index==i));
sigm(:,:,i) = cov(Data(:,index)');
end

weight = count/sum(count);
mu = center';
sigma = sigm;

-
Thanks, I actually managed to get this working but forgot to post the answer here. Your answer agrees with mine though, so it seems I was correct. Thanks! – casper Jul 9 '12 at 8:11