Tag Info

New answers tagged

3

I'm going to use an argument along the lines of what @hOff proposed. In addition to the smoothness of $G$ I am going to suppose that it decays rapidly at infinity so that all of the following calculations are justified. For example, we could assume that $G$ is Schwartz class. As the first order of business we expand your equation as $$ \Delta G(x) + x ...


0

First, let me introduce the function $f : \mathbb{R}^n \rightarrow R | f(x) = \exp( \frac{1}{2} x \cdot x)$. Since it is nowhere zero and $\frac{1}{f}\nabla f = x$, you can rewrite $\nabla G + x G = 0$ as $\nabla G + \frac{G}{f}\nabla f = 0$, which is equivalent to : $$ \nabla ( fG ) = 0 $$ Now, your second order equation reads $\Delta (fG) = ...


0

Writing $-\log(A) = \int d \beta \left[ \frac{1}{A+ \beta} - \frac{1}{1+\beta} \right] $, helps. By substituting $A \to A+\delta A$, and using binomial expansion, one gets an integral representation for the 2nd order term, which is what I was looking for. Thanks!


1

Solving the Lagrange Equations, we get that the maximum entropy distribution with mean $0$ and variance $1$ is where $$ \sum_{k\in\mathbb{Z}}(k^2-1)e^{-ak^2}=0 $$ which is $a\doteq0.4999998943842821\sim\frac12$. We need to compute the coefficient where $$ c\sum_{k\in\mathbb{Z}}e^{-ak^2}=1 $$ which is ...


1

Your intuition is correct but your calculation of entropy is not, the differential entropy of the uniform distribution on $(0,\frac{1}{2^n})$ is $$-2^n\int_0^{\frac{1}{2^n}}\log_22^ndx=-n$$ The concept of entropy for continuous random variables (differential entropy) is not well defined, since a real number requires a theoretically infinite number of bits ...


0

$\def\pp{\mathbb{P}}$The usual definition of conditional entropy of $A$ given $B$ works. It is defined as the expected conditional entropy, which is the expected weighted negative log conditional probability. This corresponds to your correct intuition since in this case the weight function is uniform since every value in $[0,1)$ is equally likely. You can ...


0

It is definitely wrong, simply because of DPI. You can check that $I(X;Y)=I(X;Z)+I(X;Y|Z)$.


1

You just carry out the chain rule as you would normally. $$ \frac{d}{dx}{\rm tr}\left[{A(x) \log A(x)}\right] = {\rm tr} [A'(x)\log A(x) + A(x)A^{-1}(x)A'(x)]$$ See here.


1

Starting with $N$ zero bits and performing $F$ flips. Let's look at a single bit. There are $(N-1)^{F-k}\binom{N}{k}$ ways that our bit can be flipped $k$ times. There are $N^F$ ways the random choices can go, so define $p_k$: $$p_k = \frac{(N-1)^{F-k}}{N^F}\binom{N}{k}$$ The generating function $G(x)$ for the number of flips is: $$G(x) = \sum_k p_kx^k ...


5

After $N=2T$ toggles, the average value of any bit is $$\sum_{k=0}^{T-1}\frac{(W-1)^{2T-2k-1}}{W^{2T}}{2T\choose 2k+1}\\ =\frac12-\frac12\left(\frac{W-2}W\right)^{2T}$$ and the second term is roughly $\frac12e^{-2N/W}$. Use enough toggles to make that difference small enough.


0

You may be looking for an edge detection and approximate your "entropy" as the total length of edges in the $0,1$-Matrix (viewed as a black and white image). The Sobel operator can help you do this. Check out $$e(M) := \| \mathcal S(M)\|_F$$ where $\mathcal S$ is the Sobel operator and $\|\cdot\|_F$ is the frobenius norm.


0

We have - being more explicit with the $p(x_i, y_j)$ terms, which are shorthand for $\def\Pr{\mathbb P}\Pr(X=x_i, Y=y_j)$, that $$ \def\p#1#2{\Pr(X_2 = #1, X_1 = #2) = \Pr(X_2 = #1 \mid X_1 = #2) \Pr(X_1 = #2)}\p ab $$ The first term can be read off from the matrix, the second one from $X_1$'s distribution. Let's start - we call the two states $0$ and ...


0

So this is a MC with $2$ states $a$ and $b$. You should start by understanding $$ p(X_2 = b | X_1 = a) $$ etc.


1

The mutual information of two normal variables with correlation factor $\rho$ is $$ I(X;Y) = -\frac{1}{2} \log(1-\rho^2) \tag{1}$$ See proof in any Information Theory textbook - eg. In our case, letting $X=A+B$ and $Y=A+C$, we have $\sigma_X^2 = \sigma_A^2+\sigma_B^2$, $\sigma_Y^2 = \sigma_A^2+\sigma_C^2$, $\sigma_{XY}=E(X Y) = \sigma_A^2$. Hence $$ ...


0

Using this definition of a restricted topology, we can state that: $$h(A∪B∪C,T|_{A∪B∪C})$$ $$=\;h(A∪B∪C,T|_A)\;+\;h(A∪B∪C,T|_B)\;+h\;(A∪B∪C,T|_C)$$ $$=\;h(A,T|_A)\;+\;h(B,T|_B)\;+\;h(C,T|_C)$$


3

It's not clearly stated what type of domain you assume, and what the distance measure is, so I'll pick a simple case of sets in $\mathbb{R}$ with the normal distance measure to provide a counter-example to your hypothesis. For $u<v$, define the probability distribution $P_{u,v}$ so that it has point probabilities $1/2$ at $u$ and $v$: i.e., probility ...



Top 50 recent answers are included