# finding the maximum likelihood estimator: conditional generalised linear model

Find the maximum likelihood estimator of $\alpha$,$\beta$ and $\lambda$ given the model \begin{equation*}P(Z=z|X=x)=\exp\bigg\{\sum_{j=1}^m(\alpha_j+\beta_j{x})z_j+\sum_{j<k}\lambda_{jk}z_jz_k-\Psi(\alpha+\beta{x},\lambda)\bigg\}\end{equation*} where \begin{equation*}\Psi (\alpha+\beta{x},\lambda)=\log\bigg\{\sum_{z\in\Omega}\exp\big\{\sum_{j=1}^m\big(\alpha_j+\beta_jx\big)z_j+\sum_{j<k}\lambda_{jk}z_jz_k\big\}\bigg\}\end{equation*} \begin{equation*}z_j=\begin{cases} 1 & \text{if success}\\ 0 & \text{if failure} \end{cases}\end{equation*} $x$ is the vector of covariates, $\alpha$,$\beta$ and $\lambda$ are the unknown parameters $\sum_{z\in\Omega}$ is the summation over all possible values $z$ can take. i was able to find the likelihood function and the log likelihood functions as follows: Denote $L$ the likelihood function of the model for a random sample of size $N$ so that we have the following\ \begin{equation*}L(\theta;z)=\prod_{i=1}^N\exp\bigg\{\sum_{j=1}^m(\alpha_j+\beta_j{x_i})z_{ij}+\sum_{j<k}\lambda_{jk}z_{ij}z_{ik}-\Psi(\alpha+\beta{x_i},\lambda)\bigg\}\end{equation*} where $\theta=(\alpha,\beta,\lambda)$

so that the log likelihood function $l$ becomes $$l=\sum_{j=1}^m\alpha_j\sum_{i=1}^Nz_{ij}+\sum_{j=1}^m\beta_j\sum_{i=1}^Nx_iz_{ij}+ \sum_{j<k}\lambda_{jk}\sum_{i=1}^Nz_{ij}z_{ik}-\sum_{i=1}^N\Psi(\alpha+\beta{x_i},\lambda)$$ my main problem is to find the maximum likelihood estimator of $\alpha$,$\beta$ and $\lambda$. I had a problem in finding the first and second partial of the log likelihood function more especially finding the partial of the $\Psi$ function. Help me find the score and fisher information.

• Would not it be easier to unterstand in matrix notation? – Caran-d'Ache Jul 16 '13 at 9:31
• @Caran-d'Ache if you can help me do it even in matrix notation I would appreciate.Thank you. – user77918 Jul 16 '13 at 9:55

## 1 Answer

Well, that's not an answer, I'll just try to help with matrix nonation. First of all let's stack all column-vectors $z_i=(z_{i1}\dots z_{im})^T$ in rows of a matrix, so that $$Z=(z_1^T,z_2^T\dots z_N^T)^T=\begin{bmatrix} z_{11} & \cdots & z_{1m} \\ \vdots & \ddots & \vdots \\ z_{N1} & \cdots & z_{Nm} \end{bmatrix}$$ Then let's put all $\lambda_{jk}$ in a matrix $\Lambda$ (note that it will be upper triangular) and all $\alpha, \beta, x$ will be combined in corresponding vectors $\vec{\alpha}, \vec{\beta}, \vec{x}$. Then your log likelihood function will look like: $$l=\vec{1}^TZ\vec{\alpha}+\vec{x}^TZ\vec{\beta}+\mathrm{Tr}(Z\Lambda Z^T)-\vec{1}^T\vec{\Psi}$$ Here $\vec{1}^T=(1,1\dots 1)$ and $\vec{\Psi}=(\Psi(\alpha+\beta{x_1},\lambda)\dots \Psi(\alpha+\beta{x_N},\lambda))^T$.
And $$\Psi(\alpha+\beta{x_i},\lambda)=\log\bigg\{\sum_{z\in\Omega}\exp\big\{\big(\vec{\alpha}^T+\vec{\beta}^Tx_i\big)z_i+\mathrm{Tr}(Z\Lambda Z^T)\big\}\bigg\}$$ Either way the last term looks very nasty, especially the summation over all possible $z$. In order to find ML estimates $\hat{\theta}=(\hat{\vec{\alpha}}, \hat{\vec{\beta}}, \hat{\vec{\lambda}})$ of $\theta=(\vec{\alpha}, \vec{\beta}, \vec{\lambda})$ one has to compute the derivatives with the respect to $\theta$, set the obtained equations to zero and solve them. So the obtained system will look like: $$\begin{cases} \frac{\partial \ l}{\partial \vec{\alpha}}=\vec{1}^TZ- \vec{1}^T\frac{\partial \vec{\Psi}}{\partial \vec{\alpha}}=0\\ \frac{\partial \ l}{\partial \vec{\beta}}=\vec{x}^TZ- \vec{1}^T\frac{\partial \vec{\Psi}}{\partial \vec{\beta}}=0\\ \frac{\partial \ l}{\partial \Lambda}=Z^TZ- \frac{\partial \ \mathrm{Tr}(\vec{1}\!\otimes\!\vec{\Psi})}{\partial \Lambda}=0 \end{cases}$$ here $\otimes$ - is the outer product. So the solution of those equations will yield you ML estimates. Do I understand you correctly that $\Omega=(0,1)$?