1
$\begingroup$

I am attempting to invert the multinomial logit link with three variables. I can do it with two variables, but I do not know how to do it with three.

A multinomial logit function for three states, i.e., three probabilities, $a, b, c$ is written as follows:

$a = \frac{e^{x}} {(1 + e^{x} + e^{y})}$

$b = \frac{e^{y}} {(1 + e^{x} + e^{y})}$

$c = 1 - a - b$

These three probabilities are defined by the parameters $x$ and $y$. If we know $x$ and $y$ we can obtain $a$, $b$ and $c$.

However, given $a, b, c$, how do we obtain $x$ and $y$? One way is to use multinomial logistic regression. However, there should be a closed form solution in which $x$ and $y$ are obtained using basic algebra. I can obtain the closed form solution for two parameters, $x$ and $y$:

$x = \log(\frac{a (1 - b) + (a b)}{ (1 - a) (1 - b) - a b})$

$y = \log(\frac{b (1 - a) + (b a) }{ (1 - b) (1 - a) - b a})$

Which simplifies to:

$x = \log(\frac{a}{1 - a - b})$

$y = \log(\frac{b}{1 - a - b})$

How can I obtain the closed form solution when there are three parameters $x, y, z\;$?

$a = \frac{e^{x}} {(1 + e^{x} + e^{y} + e^{z})}$

$b = \frac{e^{y}} {(1 + e^{x} + e^{y} + e^{z})}$

$c = \frac{e^{z}} {(1 + e^{x} + e^{y} + e^{z})}$

$d = 1 - a - b - c$

$\endgroup$
  • $\begingroup$ what is the meaning of the 3rd equation c=1-a-b? Why do you need this? The last equation c=1-a-b-c seems to be nonsense. $\endgroup$ – miracle173 Jan 7 '18 at 10:50
  • $\begingroup$ Why do you write a(1-b)+(ab) instead of a? $\endgroup$ – miracle173 Jan 7 '18 at 10:51
  • 2
    $\begingroup$ Why not work out the explicit expression for $c$ first ? for the first case $c = 1 - a - b = \frac{1}{1+e^x + e^y}$, so $e^x = \frac{a}{c} \implies \cdots$. The rest is similar. $\endgroup$ – achille hui Jan 7 '18 at 11:08
  • $\begingroup$ The very last equation should probably have a $d$, not a $c$, on the left hand side. $\endgroup$ – Barry Cipra Jan 7 '18 at 14:45
  • $\begingroup$ @BarryCipra Thank you. That was a typo probably introduced when applying the revised formatting. $\endgroup$ – Mark Miller Jan 7 '18 at 16:01
2
$\begingroup$

For every fixed number of variables, you are considering, for every $i$,

$$a_i=\frac{e^{x_i}}{1+s}$$

where

$$s=\sum_ie^{x_i}$$

and

$$z=\frac1{1+s}$$

and you are asking how to invert this system, that is, how to deduce the collection $(x_i)$ from the collection $(a_i)$ and $z$, or even, from $(a_i)$ only.

To solve this, consider that $$z+\sum_ia_i=1$$ hence, for every $i$, $$e^{x_i}=a_i\cdot(1+s)=\frac{a_i}z$$ that is,

$$x_i=\log a_i-\log z=\log\left(\frac{a_i}{1-\sum\limits_ka_k}\right)$$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Rereading this post, I see that its content was entirely, albeit more concisely, explained by @achillehui in a comment on main posted 6 hours before it, with no visible reaction or understanding from the OP. $\endgroup$ – Did Jan 8 '18 at 6:34
-1
$\begingroup$

If you want to solve $$\begin{eqnarray} a_1 &=& \frac{e^{x_1}} {1 + \sum_{i=1}^n e^{x_i}}\\ &\ldots& \tag{1}\\ a_n &=& \frac{e^{x_n}} {1 + \sum_{i=1}^n e^{x_i}}\\ \end{eqnarray} $$ with constants $$a_1,\ldots,a_n$$ and variables $$x_1,\ldots,x_n$$ then set $$ \begin{eqnarray} u_1&=&e^{x_1} \\ &\ldots& \tag{2}\\ u_n&=&e^{x_n} \end{eqnarray} $$ and substitute in $(1)$ and multiply each equation by its denominator and you have

$$\begin{eqnarray} a_1 + \sum_{i=1}^n a_1 u_i&=& u_1\\ &\ldots& \tag{3}\\ a_n + \sum_{i=1}^n a_1 u_i&=& u_n\\ \end{eqnarray} $$

This is a linear equation with variables $u_i.$ You can solve this numerically or algebraically with the well known methods for linear equations.

Then you can so the back substitution to get the $x_i.$

$$ \begin{eqnarray} x_1&=&\log{u_1} \\ &\ldots& \tag{4}\\ x_n&=&\log{u_n} \end{eqnarray} $$

| cite | improve this answer | |
$\endgroup$
-3
$\begingroup$

Solve each variable $x$, $y$ and $z$ as a function of the other two.

$x = \log(\frac{(a + a e^{y} + a e^{z})} {(1 - a)})$

$y = \log(\frac{(b + b e^{x} + b e^{z})} {(1 - b)})$

$z = \log(\frac{(c + c e^{x} + c e^{y})} {(1 - c)})$

Then:

Substitute $x$ into the equation for $y$.

Substitute $x$ into the equation for $z$.

Substitute $y$ into the equation for $x$.

Substitute $y$ into the equation for $z$.

Substitute $z$ into the equation for $x$.

And substitute $z$ into the equation for $y$.

Each parameter $x$, $y$ and $z$ is now expressed as a function of just one of the other two.

To express each parameter as a function of itself, substitute, for example, the equation expressing $x$ as a function of $y$ into the function expressing $y$ as a function of $x$.

Once these three equations are simplified we have:

$x = \log(-(a * c - a) / (c^2 + (b+a-2)*c + (a - 1)*b - a + 1 - a * b))$

$y = \log(-(b * c - b) / (c^2 + (b+a-2)*c + (a - 1)*b - a + 1 - a * b))$

$z = \log(((1-b)*c) / (((b+a-1)*c+b^2+(a-2)*b-a+1) - (a*c)))$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.