# Mutual information, handling start and end

In my homework, I am supposed to use Brown algorithm to compute word classes. I need to compute mutual information for some data and looking in my results and official results, I am probably making some error in handling beginning and end of data. Suppose i have a data "a b a c", how can i compute mutual information using formula:

$$I(X;Y) = \sum_{y \in Y} \sum_{x \in X} p(x,y) \log{ \left(\frac{p(x,y)}{p(x)\,p(y)} \right) }, \,\!$$

What will be first and last bigram and unigram and what should be result of this simple example? Thanks a lot!

-
Usually one adds additional start end tokens. So, in your example, if "_" does not appear in your vocabulary, one can use that to indicate start end token; thereby your data then would look like this "_ a b a c _" and your bi-grams would look like "_ a", "a b", ..., "c _". Note, if you are working with n-grams, you will have to add n - 1 "_" at the start and at the end. For uni-grams, one need not add the additional start end tokens. –  TenaliRaman Jun 20 '12 at 22:19
Thanks for answer, it helped me! –  Ondra Jun 22 '12 at 12:26