83 reputation
19
bio website
location
age
visits member for 3 years, 7 months
seen Jun 24 at 6:48

Feb
25
comment How could we know the Dirichlet allocation is describing the topic rather than something else?
I don't have much reputation points. I will wait another day before I start a bounty.
Feb
23
comment Conjugate priors make calculations easier but at what cost to the model?
Thanks, it's more formal now.
Feb
6
comment Is there any software package to calculate the entropy, information content, mutual information, etc?
That looks awesome. Could you re-post your 2 comments as an answer so I can mark it?
Feb
6
comment Is there any software package to calculate the entropy, information content, mutual information, etc?
Thanks. If you change your comment to an answer, I will mark it. BTW, is there any Python option?
Feb
4
comment What does the -log[P(X)] mean in the calculation of entropy?
It does reveal something. Thanks. Actually, since the entropy/self-information is used to quantify/measure the uncertainty of a random variable, I did think about selecting a mathematical function according to a set of preferred properties. These properties are based on some common sense about the relation between uncertainty and information gain. It seems Shannon also took this approach.
Feb
4
comment What does the -log[P(X)] mean in the calculation of entropy?
From the perspective of probability, the entropy is just an expectation of -log[P(i)], so -log[P(i)] must have some solid meaning. So what is it?
Feb
4
comment What does the -log[P(X)] mean in the calculation of entropy?
It's an interesting/intuitive explanation when you mention "the surprise in n tosses is n×(surprise in one toss)"
Feb
2
comment Is conditional probability also probability?
Much appreciate your supplement. I will take some time to digest them. (BTW, is there any free download of Renyi's thesis?)
Feb
2
comment Is conditional probability also probability?
I haven't read Renyi's article yet. But indeed, I did considered that conditional probability should be more general. And probability is inherently a relative concept. Since it is human nature to consider things in a relative way. When we estimate the probability of something, we are actually (implicitly) compare the possibility of that thing to the possibility of something we are certain about. It is through such comparison that we get some feeling/estimation about the degree of possibility/belief.
Feb
2
comment Is conditional probability also probability?
When a deadend is encountered, we should look back to see if the start point is incorrect/too limited. This seems to be a promising answer. I will read that article when back home. Thanks!
Feb
1
comment Is conditional probability also probability?
Thanks. This is indeed an vivid/reasonable example. It could be seen as we changed to a new universe when considering the condition. But how could we use the value from the old universe in the new universe?
Feb
1
comment Is conditional probability also probability?
Yes, my doubt is about the quotient being of the same type as the divisor and divided. I can think ratio as the ultimate/subconscious origin of probability, but in some context, the ratio is not calculatable, such as the degree-of-belief scenario. But we just generalize our theory for all scenarios.
Jan
31
comment Is conditional probability also probability?
I just take a brief look at the Ch.9 of that book. It seems I need to read it from the start to comprehend it. It could take a while. Could you summarize a bit about the point?
Jan
31
comment Is conditional probability also probability?
Thanks for giving me a hint. I will check that book and get back to you later.
Jan
31
comment Is conditional probability also probability?
(i) An interesting explanation from the linguistic perspective. (ii) Indeed I verified this before according to the 3 axioms. But is it enough for justification?
Jan
31
comment Is conditional probability also probability?
I updated my question.
Jan
31
comment Is conditional probability also probability?
Thanks. By the sister example, I am referring to the degree-of-belief scenario where probability also arises.
Jan
31
comment Is conditional probability also probability?
I agree that we need to change the sample set S to a smaller one specified by the condition. But after that, we should get the probability P'(A) the same way as we get the P(A), rather than simply calculate the ratio of P(A∩B)/P(B).
Jan
31
comment Is conditional probability also probability?
@user73985 I took the μ(S)=1 before just as you did. But I give it up because of the limit of ratio interpretation.
Jan
31
comment Is conditional probability also probability?
@nomen I think you mean probability by ratio.