Tagged Questions

1
vote
0answers
90 views

Mutual Information of Correlated Bivariate Uniform Distribution

We have correlated bivariate uniform distribution, where X and Y have a correlation coefficient $\rho$ and they uniformly distributed in the following rectangle. What is the mutual information of $X$ ...
2
votes
1answer
43 views

Parameter optimization in probabilistic models

Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given ...
0
votes
1answer
69 views

Test for randomness

I'm trying to write a program to compute a metric for the entropy in files to determine a probability that the file is compressed or encrypted. Compressed and encrypted files have very, very, very ...
0
votes
1answer
160 views

Relative Entropy given two non-equivalent sets

I am trying to calculate the relative entropy given two collections and have a question regarding some issues. Supposed we have two sets, $Real$ and $Calculated$, and their respective probability ...