0
$\begingroup$

Define the entropy of a random variable $X$ by : $$H(X):=\sum_{x\in X(\Omega)}p(x)\log\left(\frac{1}{p(x)} \right), $$ with $0\log 0:=0$ and $p(x)=P(X=x)$.

A fundamental propriety satisfied by $H$ is : $$H(X)=\log |range(X)|, $$ if and only if $X$ is a uniform random variable.

Let $E$ be a finite-dimensional space vector over a finite field $\mathbb{F}$, then and $E^{*}$ the dual space. Then, if $X$ is a uniform variable over $E^{*}$ : $$H(X)=\log |range(X)|=\log |E^{*}|=\log |\mathbb{F}^{\text{dim}(E)}|)=\text{dim}(E)\log|\mathbb{F}|.$$

My question is : if $U,V$ are two linear subspaces of $E$ an $X,Y$ two uniform variables over $U$ and $V$, how do you prove that : $$H(X,Y)=\text{dim}(U+V)\log|\mathbb{F}|. $$

Note : In the following article

https://terrytao.wordpress.com/2017/03/01/special-cases-of-shannon-entropy/

Tao wrote that "the joint random variable $(X,Y)$ determines the random linear function $f$ $\color{red}{\text{on the union } U \cup V}$ on the two spaces, and $\color{red}{\text{thus by linearity on the Minkowski sum }U+V}$ as well; thus $(X,Y)$ is equivalent to the restriction of $f$ to $U+V.$ In particular, $H(X,Y) = \mathrm{dim}(U+V) \log |\mathbf{F}|$.",

but I don't really understand the 'red' argument.

$\endgroup$
4
  • 1
    $\begingroup$ You left out a key part from Tao's writing. $X$ and $Y$ are not just any random variables. $X$ is gotten by restricting a random function $f\in E^*$ to the subspace $U$, and similarly $Y$ is gotten by restricting the same random function to $V$. Consequently the components $(X,Y)$ automatically agree on $U\cap V$, and thus yield a well defined linear function on $U+V$. $\endgroup$
    – Jyrki Lahtonen
    Commented Jun 19, 2019 at 6:06
  • $\begingroup$ It's ok for the definition of $X$ and $Y$, but I don't understand the link between '$(X,Y)$ automatically agree on $U\cap V$' and it defines a 'linear function on $U+V$'. $\endgroup$
    – Mishikumo2019
    Commented Jun 19, 2019 at 7:11
  • 1
    $\begingroup$ In the beginning Tao has a random linear function $f:E\to\Bbb{F}$. Then $X$ is the restriction of $f$ to the subspace $U$. Similarly $Y$ is the restriction to the subspace $V$. Therefore the restrictions of both $X$ and $Y$ to the subspace $U\cap V$ agree. Given two linear function $X:U\to\Bbb{F}$ and $Y:V\to\Bbb{F}$ that agree on $U\cap V$, we can define a unique linear function $(X,Y)$ from $U+V$ to $\Bbb{F}$ by declaring $(X,Y)(u+v)=X(u)+Y(v)$. This is well-defined even though we may have $u+v=u'+v'$ with $u'\neq u$ and $v\neq v'$. $\endgroup$
    – Jyrki Lahtonen
    Commented Jun 19, 2019 at 10:06
  • 1
    $\begingroup$ (cont'd) This is because $u-u'=v'-v$ is an element of $U\cap V$, so $X(u-u')=Y(v'-v)$. My understanding is that this may have been your problem, but I'm prepared to be wrong. $\endgroup$
    – Jyrki Lahtonen
    Commented Jun 19, 2019 at 10:08

1 Answer 1

Reset to default
0
$\begingroup$

Solution based on @Jyrki Lahtonen comments :

Let $X\sim\mathcal{U}(E^{*})$ and $X_1=X_{|U}$ and $X_2=X_{|V}$. These two functions are equal on $U\cap V$, and we can define a unique function $(X,Y)$ from $U+V$ to $\mathbb{F}$ by declaring : $$(X,Y)(u+v)=X(u)+Y(v). $$ This function is well defined because if $u+v=u'+v'$, $(X,Y)(u+v)=(X,Y)(u'+v')$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .