Held out estimator is an empiric estimation technique for calculating probabilities of events. (I would put a Wikipedia link, but i couldn't find a wikipedia page on this subject)

The main idea is to split the experiments data into two sets: Training and Held-out. (from this point and onward i will be using the notion of NLP for clarity)

Define: $S^T$ - The training corpus
$S^H$ - The calibration corpus (held-out)
$N_r^T$ - The number of events that appear in $S^T$ $r$ times
$c^T(x)$,$C^H(x)$ - The number of time that event $x$ appears in $S^T$ and $S^H$, respectfully.

According to the notations above, the calculation of the probability of an event, measured by the HeldOut estimator is given by the following equation:

$P_{HO} (x:c^T (x)=r)=\frac{∑_{x:c^T (x)=r}c^H (x)}{N_r^T|S^H |}$

And now for the question...

This is all good and nice if the event we are talking about is an event that appears both in $S^T$ and in $S^H$, but what happens when $C^T(x)=C^H(x)=0$? These are words (events) that don't appear in any of the corpora, so I don't know what words they are. If so, how can I get $C^H(x)$ if I have no Idea who $x$ is in the first place?

Is there some other equation or something that is the calculation for unseen words(events)?

Am i missing something? Is there another equation or something that deals with unseen words?


1 Answer 1


Getting the number of events that are not in the Held-Out corpus (denoted $C^H(x)=0$) requires prior knowledge of the number of event in the world. After knowing the size of the events group (how many event exist in the probability space) you can simply subtract the number of actual unique events in the corpus from that number and get the required result.

Example: If our corpus has 20 unique events and there are 30 events that may have occurred, then $C^H(x)=30-20=10$


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .