2
$\begingroup$

I would say that the function $f(x)=x^2$ has more "information" than the function $f(x)=1$. Is this true, and if so, is there a way to measure the information content of a function?

$\endgroup$
0
$\begingroup$

I would say that information is a relative thing. You need to define a reference space, so you can talk probabilities. When you have probabilities, then you can measure information.

If your space contains only the two mentioned functions, then you need to make a single measurement at $x=0$ to decide which function is presented. So, the functions are equal and you will need a single bit to describe the space.

You may have a different space though. For instance, take the same functions, plus a few new functions that are exactly similar to $f(x)=x^2$, on an interval $[a,b]$ and mutually different, on the rest of the domain. Then, we need more information to pinpoint $f(x)=x^2$, compared to $f(x)=1$, as there are some similar functions.

$\endgroup$
0
$\begingroup$

To be concrete, let $f:[-N,N]\rightarrow \mathbb{R}.$ The usual way of defining the distribution of a random variable is how we can proceed. Assume the uniform measure on $[-N,N]$ with $N<\infty$, and let $f(X)=Z.$ Then $$\mathbb{P}(Z \in A)=\mathbb{P}(f(X) \in A)=\mathbb{P}(f^{-1}(A)).$$

In this context we can say that the entropy of the function is the entropy of its output as a random variable.

If we choose the $f(X)=c,$ some constant then clearly $H(Z)=H(f)=0,$ since all the probability is concentrated at one point.

If $f$ is one-to-one, then the entropy is maximal.

If $f$ is two-to-one, then the entropy is smaller $(f(x)=x^2).$

Since it is the entropy of the function which maps the set $[-N,N]$ I wouldn't put any other measure on $[-N,N]$ and that's why I took $N<\infty$ to ensure the uniform distribution exists.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.