Apologies in advance for any misused terminology, or if this is the wrong place for the question (I think it's okay though).

I am given a group of logarithmic expressions such as:

  • $- (a \log(a) + (1 - a)\log(\frac{1-a}{2}))$
  • $- (b \log(b) + (1 - b)\log(\frac{1-b}{4}))$
  • $- (c \log(c) + (1 - c)\log(\frac{1-c}{5}))$

My goal is to find values of $a$, $b$, and $c$ that make these expressions equal.

The above is just an example of the sort of expressions I get, but they are always of the form $-(x\log(x) + (1-x) log(\frac{1-x}{n}))$, where $n$ is a an integer $\ge 1$, and $x$ is some symbol that doesn't occur in more than one expression in the group.

Ultimately I'll be using a computer to solve these problems. But my question is: Can I solve this by hand? And if so, how? Alternatively, can I distill the problem down to something simpler to code up?

I imagine I'd start by creating a set of equations that expresses equality between the expressions. So between expression 1 and 2 I made:

$- (a \log(a) + (1 - a)\log(\frac{1-a}{2})) = - (b \log(b) + (1 - b)\log(\frac{1-b}{4}))$

But I'm not sure how to go about solving for the values of the variables. I found a bunch of info on solving systems of linear equations but I'm not sure what to do with the logs.

  • $\begingroup$ Just to clarify before I attempt to answer - $\log$ refers to the natural logarithm, yes? $\endgroup$ – Zubin Mukerjee Dec 2 '13 at 0:32
  • 1
    $\begingroup$ @ZubinMukerjee: I can assure you that it makes no difference at all since $\log_a(x)=k\cdot\ln(x)$ for some constant $k$. In fact this constant is $k=\log_a(\mbox{e})=1/\ln(a)$. This is known as 'change of base for logarithms'. $\endgroup$ – String Dec 2 '13 at 0:38
  • $\begingroup$ I think he was just asking for clarification before writing up his solution. $\endgroup$ – Doc Dec 2 '13 at 0:40
  • $\begingroup$ I'm assuming base 2. The expressions were derived from some information entropy equations. Each expression is the entropy of some distribution. (This has made me realize some errors, and I've changed the expressions accordingly.) $\endgroup$ – oadams Dec 2 '13 at 0:53
  • $\begingroup$ I see that the expressions only make sence for $a,b,c,x,...\in(0,1)$. Also if $x+y=1$ the general form of the expressions is $$x\log(x)-y\log(y)+y\log(n)$$ $\endgroup$ – String Dec 2 '13 at 1:02

I wrote this when the expressions in the question didn't have leading negatives. However, I think the method still works in the same way.

This solution requires computation that cannot be done (or at least would be very tedious) by hand.

Let's simplify the general form of the expressions you're setting equal to each other:

$$x \log(x) - \left(1-x\right))\log\left(\frac{1-x}{n}\right)$$

$$x \log(x) - \left(1-x\right)\left(\log\left(1-x\right)-\log(n)\right)$$

Note that for both $\log(x)$ and $\log(1-x)$ to be real numbers, $x$ must satisfy $0 < x < 1$. We'll continue with this assumption.

$$x\log(x) - \left(1-x\right)\log\left(1-x\right) + \left(1-x\right)\log\left(n\right)$$

Set this equal to a constant $k$:

$$x\log(x) - \left(1-x\right)\log\left(1-x\right) + \left(1-x\right)\log\left(n\right) = k$$

$$x\log(x) - \left(1-x\right)\log\left(1-x\right) = \left(x-1\right)\log\left(n\right)+k\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,(1)$$

This is a graph (from Wolfram Alpha) of the left side of the above equation, strictly between $x=0$ and $x=1$:

We want to find a fixed value of $k$ for which there exist values of $x$, call them $x_n$ (in your question, these were the $a$, $b$, $c$, etc.) for every integer $n \geq 1$, so that equation $(1)$ holds.

In particular, this value of $k$ must yield a solution when $n=1$. When $n=1$, $\log(n) = \log(1) = 0$, so $\left(x-1\right)\log\left(n\right)+k$ is just the horizontal line given by $y=k$. This means $k$ must be between approximately $-0.14$ and $0.14$ in order for there to be a solution to equation $(1)$.

When $n>1$, $y=(x-1)\log(n) + k$ is a line with positive slope that intersects the $y$-axis at $y=k-\log(n)$.

I propose that $k=0.1$ yields a solution $x_n$ for all integers $n \geq 1$.

This is a graph of the right side of equation $(1)$, superimposed on the graph of the left side, when $n = 2$ and $k = 0.1$:

enter image description here

Note that the line $y=(x-1)\log(n) + 0.1$ intersects the vertical line $x=1$ at $y=0.1$, regardless of what we choose $n$ to be.

My graphical skills are poor, so you'll have to imagine the set of lines $y=(x-1)\log(n) + 0.1$ over all integers $n \geq 1$ as a set of lines that all have non-negative slope and all pass through the point $(x,y) = (1,0.1)$.

It should be clear that there must be an $x_n$ that solves equation $(1)$ for each $n$.

The value we chose for $k$, $0.1$, was certainly not the only one. Any $k$ in some neighborhood of $k=0.1$ would yield a set of $x_i$ that work with equation $(1)$.

This implies that the solutions to your infinite system of equations are nowhere close to unique. When setting all of the equations equal to each other, I would recommend choosing a value of $k$ to set them all equal to, e.g. $k=0.1$.

Finding the values of $x_n$, once you've fixed a $k$ shouldn't be too difficult to code. For each $n$, finding $x_n$ is finding the intersection of a line that goes through $(x,y) = (1, k)$ with slope $\log(n)$, with the curve $y=x\log(x) - \left(1-x\right)\log\left(1-x\right)$.

I hope that helps.

  • $\begingroup$ Thank you for this thorough and instructional answer. Although I unfairly changed the sign not just at the start of the expression, but in the middle as well, the principles here showed me what to do nonetheless. $\endgroup$ – oadams Dec 2 '13 at 6:11

$$x \log(x) - (1-x) \log\left(\frac{1-x}{n}\right) = \log \left(x^x \left(\frac{n}{1-x}\right)^{1-x}\right)$$ but that's about all the simplification you'll get. In particular, you're unlikely to get closed-form solutions when equating these for different $x$'s and different $n$'s.


Disclaimer: This is nothing near a solution - just an observation:

Here is a dynamic graph (for changing values of $n$) of $$ f_n(x)=f_1(x)+(1-x)\log(n) $$ where $f_1(x)=x\log(x)-(1-x)\log(1-x)$:

enter image description here

As can be seen above the graph of $f_1(x)$ is simply added to a line through $(0,\log(n))$ and $(1,0)$.

  • $\begingroup$ I think this puts much more clearly the part in my answer where I say "imagine the set of lines ..." :) $\endgroup$ – Zubin Mukerjee Dec 2 '13 at 1:37
  • $\begingroup$ @ZubinMukerjee: I did not add in much brain-work, though :) Great observations on your part! I just programmed it into GeoGebra to consider what happens when $n$ changes... $\endgroup$ – String Dec 2 '13 at 1:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.