Suppose I have a set A of n numbers which correspond to probabilities of actions in an environment. In other words, $$ \sum_{i=1}^{i=n} a_i = 1 $$ and $$\forall a_i = [0,1]$$ These probabilities are fixed, i.e. they never change. Now, I want to take the top k most likely items from A, where k < n, rescale their probabilities so they sum to 1, and use them to sample actions in this environment. For instance, if k = 20, $$B_{20} ⊆ A$$ $$ \sum_{i=1}^{i=20} b_i = 1 $$ $$\forall b_i = [0,1]$$ Now, the probability of selecting action $b_i$ through sampling is equivalent to its value. What problems could I run into with this approach? Do I need to assume that the probabilities in A follow a certain distribution for this to work? Do I have other options than rescaling with a softmax function? My intuition is that the probabilities of the top k actions in B might be closer together than the probabilities of the same actions in A.
$\begingroup$
$\endgroup$
6
-
1$\begingroup$ This is a bit vague, but I believe that all you are doing is speaking of conditional probabilities. That is, you condition on the event "the observed result is among the top $k$ most likely results". You don't need to assume anything about the distributions. Does that answer your question or have I misunderstood? $\endgroup$– luluApr 6 at 11:05
-
$\begingroup$ I understand these are conditional probabilities, but I am concerned about not preserving the probability properties that actions had in A, when I sample from them in B. In other words, is the relative distance between the topk samples in B the same as the relative distance between the topk samples in A? If not, what consequences can be drawn from this? $\endgroup$– postnubilaphoebusApr 6 at 11:13
-
1$\begingroup$ I don't know what topk means. Nor do I know what the "relative distance" between events in a probability space is. A priori, the $a_i$ might be non-quantifiable results. Like the first letter in someone's last name. $\endgroup$– luluApr 6 at 11:16
-
$\begingroup$ My apologies, with top k I mean the argmax of A, selecting for the highest k probabilities. With preserving relative distance I mean the following (for example): Assume the probabilities of action 0 and 1 relate to each other in A s.t.: a_0 = 2 * a_1. I would like to preserve this property in B, i.e.: b_0 = 2 * b_1 for all k probabilities in B. $\endgroup$– postnubilaphoebusApr 6 at 11:19
-
1$\begingroup$ For conditional probabilities, you are just dividing by the probability that the conditioning event occurs. And indeed the ratios you describe are preserved if you divide both sides by a fixed number. $\endgroup$– luluApr 6 at 11:21
|
Show 1 more comment