Multinomial Logistic Regression likelihood

Question

Suppose we have the following parametric model for logistic regression: $$\phi_{i} = \frac{\exp{(a^{T}x_{[i]}})}{\sum_{k = 1} ^{M} \exp{(a^{T}x_{[k]})}}$$for $i = 1, \dots, M$ and that the parameter set $X = \{x_{[i]}: i = 1, \dots, M \}$ consists of $M$ vectors $x_{[I]} \in \mathbb{R}^{n}$ we want to calibrate.

I want to find the log-likelihood for this model and the associated maximisation problem. Further, I want to show that this is a concave function.

I am however very confused as to how I should proceed. I believe I should introduce some vectors $y_{j}$ that should represent the class the observation should be in, e.g. $y_{j} = e_{i}$ representing that $a_{j}$ should be in class $C_{i}$. I am though not sure how to introduce these in the calculations.

Answer 1

you need to maximize the product of the probability function $\prod_{i=1}^M \phi_i $
Maximize with the log as log function will have the same optimal points $x_i$ as original function. $\log L = \sum_i \log \phi_i$
Now $\log \phi_i = \log (\exp{a^{T}x_{[i]}}) = a^{T}x_{[i]} - \log \sum_k \exp (a^Tx_k)$\

So max Log L = $\sum_i a^{T}x_{[i]}-M{\sum_{k = 1} ^{M} (\log \exp(a^{T}x_{[k]})})$

but I think parametric model will be $\phi_i = {{e^{-a^Tx_i}}\over{e^{\sum_k -a^Tx_k}} }$

So $\log \phi_i = \sum_k^M a^Tx_k- a^Tx_i $