Reputation
27,464
Next tag badge:
99/100 score
27/20 answers
Badges
3 27 76
Newest
 Good Answer
Impact
~814k people reached

3h
comment Relationship between sum of element of a matrix and its inverse
Related questions (few answers): math.stackexchange.com/questions/488296/… math.stackexchange.com/questions/1063551/… mathoverflow.net/questions/45087/… math.stackexchange.com/questions/1432221/…
6h
comment Relationship between sum of element of a matrix and its inverse
@Aman I guess that $e=(1,1 \cdots 1)$ so that $A$ is the sum of the elements of $M$
6h
comment Seeking help with an error function Integral
As $x \to -\infty$ the first factor inside the integral tends to $+\infty$ and the second to $-2$. This smells wrong to me. Isn't some bad sign somewhere?
1d
revised Sampling from Bivariate Normal, Relation between two coordinates
added 44 characters in body
May
2
comment Intuitive explanation of Shannon's source entropy in information / communications theory
@SKM Yes, that's basically the first Shannon theorem. You need (on average) at least $H$ bits to encode a source (not a message!) of entropy $H$. en.wikipedia.org/wiki/Shannon%27s_source_coding_theorem (BTW: you expression "a source of length $N$" smells bad - a source has no length)
May
1
comment Intuitive explanation of Shannon's source entropy in information / communications theory
You don't apply the Shannon entropy to "calculate the number of bits required to encode a message." You apply it to calculate (or rather bound) the average number of bits required to encode a source that emits random messages according to some known probability distribution. Shannon entropy is a probabilistic concept, and it requires a probabilistic model to be applied.
May
1
comment Why do we like sticking random variables into their own distributions?
"without foresight, $f(X)$ seems like a strange object to consider". I agree. To compute the entropy we compute the expectation of a thing that involves the density itself, and that's rather unusual. I'm not sure on your conclusion that "$f(X)$ is extremely fundamental".
Apr
29
comment Sending bits and parity bits over noisy channel
What matters is the probability of getting the original bits right, for computing that you need to specify how you plan to use the parity bits to decode. Of course, in general (for a rational coder-decoder pair) adding redundacy (parity bits) should decrease the overall probabiblity of error.
Apr
28
awarded  Good Answer
Apr
27
comment Conditional entropy under quantization
@Bernhard I'm ok with that. But, then, the (Shannon) entropy $H(Y|X=x) $ with $Y$ continuous is $+\infty$ unless the density consists of Dirac deltas (i.e. unless $Y$ is discrete - informally speaking).
Apr
26
comment Conditional entropy under quantization
@Bernhard What is $H(Y|X)$ when $Y$ is continuous? It's a differential entropy or not? How do you define it?
Apr
26
revised Computing mean of probability density function without integration
added 63 characters in body
Apr
26
answered Computing mean of probability density function without integration
Apr
26
answered A fair coin is continually flipped until heads appears for the 10th time. Find the number of expected tails
Apr
26
comment Computing mean of probability density function without integration
Have you plotted it?
Apr
22
comment Channel capacity of sum of symmetric channels
"so they now have different input alphabets but the same output alphabet" That's not very clear for me. Could you elaborate?
Apr
20
revised What is a probability that at 2nd turn you will pick green ball?
added 8 characters in body
Apr
18
comment number of ways to put N labelled balls in N labelled boxes so that labels don't match?
This is known as derangement en.wikipedia.org/wiki/Derangement
Apr
17
revised proof of upper bound on differential entropy of f(X)
added 2 characters in body
Apr
11
revised Is stating $\operatorname{E}(x) = 0, \operatorname{cov}(y,x)=0$, the same as stating $\operatorname E(y\,|\,x)=0$?
added 28 characters in body