I am modeling the probability of generating a sentence of dictionary words with a probability distribution over letter frequencies and word lengths.

For instance, a 10 word sentence is expected to have 2 two letter words, 2 three letter words, 2 four letter words, 1 five letter word, etc.

Using the letter frequencies, the two letter words have a 6.6% chance of being in the dictionary, the three letter words have a 2.3% chance, etc.

So, to calculate the probability that a 10 word sentence has all dictionary words I find the probability of all the k letter words being in the dictionary, and multiply these probabilities together: $0.066^2 \times 0.023^2 \times ... = 8.4 \times 10^{-33}$

Now, I can generalize this by turning it into a probability that a single word is in the dictionary when generated from the letter and length frequency distributions. $P[w \in Dict | Ltr, Len] = \sum P[w_k \in Dict | Ltr]^{P[w \in Len_k]}$.

However, this notation looks weird. I have not seen one distribution exponentiated by another distribution before. Is there a standard notation for what I am trying to do here?

  • 2
    $\begingroup$ When you say add probabilities do you mean multiply? $\endgroup$ – Kitter Catter Aug 11 '16 at 15:53
  • $\begingroup$ Also when you exponentiate shouldn't that be to the number of words power rather than to the probability power? with the probability multiplying the other probability $\endgroup$ – Kitter Catter Aug 11 '16 at 15:55
  • $\begingroup$ Yes, by add I mean multiply. $\endgroup$ – yters Aug 11 '16 at 16:07
  • $\begingroup$ The probability times the sentence length gives the number of words. So, for a sentence of a single word it is just the probability. I'm sure I'm using the wrong term here, but that's best as I can explain it. $\endgroup$ – yters Aug 11 '16 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.