Let's say that I have a problem of linear classification with $K$ classes. The logistic regression model is:

$$\log\frac{P[Class=1|X =x]}{P[Class = K|X=x]} = \beta_{10}+\beta_1^Tx$$ $$...$$ $$\log\frac{P[Class=K-1|X =x]}{P[Class = K|X=x]} = \beta_{(K-1)0}+\beta_{K-1}^Tx$$

so I have :

$$ 1) P[Class=k | X=x] = \frac{e^{\beta_{k0}+\beta_k^Tx}}{1+\sum_{l=1}^{K-1}e^{\beta_{l0}+\beta_l^Tx}}$$ $$2) P[Class=K | X=x] = \frac{1}{1+\sum_{l=1}^{K-1}e^{\beta_{l0}+\beta_l^Tx}}$$

I am not able to understand how to calculate the last two probabilities $1)$ and $2)$. Could you give me any advice on how I should do it?


1 Answer 1


Using the abbreviation $p_i := P(\text{class}=i\mid X=x)$, for $i=1,\ldots, K$, verify that

$$1-p_K = \sum_{i=1}^{K-1} p_i = p_K\sum_{i=1}^{K-1} \frac{p_i}{p_K}.\tag a$$ Rearrange (a) to obtain: $$ p_K=\frac1{1+\sum_{i=1}^{K-1} \frac{p_i}{p_K}}.\tag b $$ Next, plug the assumptions $$\frac{p_i}{p_K} = \exp(\beta_{i,0}+\beta_i^Tx)\quad\text{for $i=1,\ldots K-1$}\tag c$$ into (b) to obtain desired expression (2) for $p_K$, and plug (2) into (c) to obtain desired expressions (1) for each $p_i$.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .