0
$\begingroup$

Define the entropy of a random variable $X$ by : $$H(X):=\sum_{x\in X(\Omega)}p(x)\log\left(\frac{1}{p(x)} \right), $$ with $0\log 0:=0$ and $p(x)=P(X=x)$.

A fundamental propriety satisfied by $H$ is : $$H(X)=\log |range(X)|, $$ if and only if $X$ is a uniform random variable.

Let $E$ be a finite-dimensional space vector over a finite field $\mathbb{F}$, then and $E^{*}$ the dual space. Then, if $X$ is a uniform variable over $E^{*}$ : $$H(X)=\log |range(X)|=\log |E^{*}|=\log |\mathbb{F}^{\text{dim}(E)}|)=\text{dim}(E)\log|\mathbb{F}|.$$

My question is : if $U,V$ are two linear subspaces of $E$ an $X,Y$ two uniform variables over $U$ and $V$, how do you prove that : $$H(X,Y)=\text{dim}(U+V)\log|\mathbb{F}|. $$

Note : In the following article

https://terrytao.wordpress.com/2017/03/01/special-cases-of-shannon-entropy/

Tao wrote that "the joint random variable $(X,Y)$ determines the random linear function $f$ $\color{red}{\text{on the union } U \cup V}$ on the two spaces, and $\color{red}{\text{thus by linearity on the Minkowski sum }U+V}$ as well; thus $(X,Y)$ is equivalent to the restriction of $f$ to $U+V.$ In particular, $H(X,Y) = \mathrm{dim}(U+V) \log |\mathbf{F}|$.",

but I don't really understand the 'red' argument.

$\endgroup$
4
  • 1
    $\begingroup$ You left out a key part from Tao's writing. $X$ and $Y$ are not just any random variables. $X$ is gotten by restricting a random function $f\in E^*$ to the subspace $U$, and similarly $Y$ is gotten by restricting the same random function to $V$. Consequently the components $(X,Y)$ automatically agree on $U\cap V$, and thus yield a well defined linear function on $U+V$. $\endgroup$ Jun 19 '19 at 6:06
  • $\begingroup$ It's ok for the definition of $X$ and $Y$, but I don't understand the link between '$(X,Y)$ automatically agree on $U\cap V$' and it defines a 'linear function on $U+V$'. $\endgroup$ Jun 19 '19 at 7:11
  • 1
    $\begingroup$ In the beginning Tao has a random linear function $f:E\to\Bbb{F}$. Then $X$ is the restriction of $f$ to the subspace $U$. Similarly $Y$ is the restriction to the subspace $V$. Therefore the restrictions of both $X$ and $Y$ to the subspace $U\cap V$ agree. Given two linear function $X:U\to\Bbb{F}$ and $Y:V\to\Bbb{F}$ that agree on $U\cap V$, we can define a unique linear function $(X,Y)$ from $U+V$ to $\Bbb{F}$ by declaring $(X,Y)(u+v)=X(u)+Y(v)$. This is well-defined even though we may have $u+v=u'+v'$ with $u'\neq u$ and $v\neq v'$. $\endgroup$ Jun 19 '19 at 10:06
  • 1
    $\begingroup$ (cont'd) This is because $u-u'=v'-v$ is an element of $U\cap V$, so $X(u-u')=Y(v'-v)$. My understanding is that this may have been your problem, but I'm prepared to be wrong. $\endgroup$ Jun 19 '19 at 10:08
0
$\begingroup$

Solution based on @Jyrki Lahtonen comments :

Let $X\sim\mathcal{U}(E^{*})$ and $X_1=X_{|U}$ and $X_2=X_{|V}$. These two functions are equal on $U\cap V$, and we can define a unique function $(X,Y)$ from $U+V$ to $\mathbb{F}$ by declaring : $$(X,Y)(u+v)=X(u)+Y(v). $$ This function is well defined because if $u+v=u'+v'$, $(X,Y)(u+v)=(X,Y)(u'+v')$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.