Find the maximum likelihood estimator of $\alpha$,$\beta$ and $\lambda$ given the model \begin{equation*}P(Z=z|X=x)=\exp\bigg\{\sum_{j=1}^m(\alpha_j+\beta_j{x})z_j+\sum_{j<k}\lambda_{jk}z_jz_k-\Psi(\alpha+\beta{x},\lambda)\bigg\}\end{equation*} where \begin{equation*}\Psi (\alpha+\beta{x},\lambda)=\log\bigg\{\sum_{z\in\Omega}\exp\big\{\sum_{j=1}^m\big(\alpha_j+\beta_jx\big)z_j+\sum_{j<k}\lambda_{jk}z_jz_k\big\}\bigg\}\end{equation*} \begin{equation*}z_j=\begin{cases} 1 & \text{if success}\\ 0 & \text{if failure} \end{cases}\end{equation*} $x$ is the vector of covariates, $\alpha$,$\beta$ and $\lambda$ are the unknown parameters $\sum_{z\in\Omega}$ is the summation over all possible values $z$ can take. i was able to find the likelihood function and the log likelihood functions as follows: Denote $L$ the likelihood function of the model for a random sample of size $N$ so that we have the following\ \begin{equation*}L(\theta;z)=\prod_{i=1}^N\exp\bigg\{\sum_{j=1}^m(\alpha_j+\beta_j{x_i})z_{ij}+\sum_{j<k}\lambda_{jk}z_{ij}z_{ik}-\Psi(\alpha+\beta{x_i},\lambda)\bigg\}\end{equation*} where $\theta=(\alpha,\beta,\lambda)$

so that the log likelihood function $l$ becomes \begin{equation}l=\sum_{j=1}^m\alpha_j\sum_{i=1}^Nz_{ij}+\sum_{j=1}^m\beta_j\sum_{i=1}^Nx_iz_{ij}+ \sum_{j<k}\lambda_{jk}\sum_{i=1}^Nz_{ij}z_{ik}-\sum_{i=1}^N\Psi(\alpha+\beta{x_i},\lambda)\end{equation} my main problem is to find the maximum likelihood estimator of $\alpha$,$\beta$ and $\lambda$. I had a problem in finding the first and second partial of the log likelihood function more especially finding the partial of the $\Psi$ function. Help me find the score and fisher information.

  • $\begingroup$ Would not it be easier to unterstand in matrix notation? $\endgroup$ – Caran-d'Ache Jul 16 '13 at 9:31
  • $\begingroup$ @Caran-d'Ache if you can help me do it even in matrix notation I would appreciate.Thank you. $\endgroup$ – user77918 Jul 16 '13 at 9:55

Well, that's not an answer, I'll just try to help with matrix nonation. First of all let's stack all column-vectors $z_i=(z_{i1}\dots z_{im})^T$ in rows of a matrix, so that $$Z=(z_1^T,z_2^T\dots z_N^T)^T=\begin{bmatrix} z_{11} & \cdots & z_{1m} \\ \vdots & \ddots & \vdots \\ z_{N1} & \cdots & z_{Nm} \end{bmatrix} $$ Then let's put all $\lambda_{jk}$ in a matrix $\Lambda$ (note that it will be upper triangular) and all $\alpha, \beta, x$ will be combined in corresponding vectors $\vec{\alpha}, \vec{\beta}, \vec{x}$. Then your log likelihood function will look like: $$l=\vec{1}^TZ\vec{\alpha}+\vec{x}^TZ\vec{\beta}+\mathrm{Tr}(Z\Lambda Z^T)-\vec{1}^T\vec{\Psi}$$ Here $\vec{1}^T=(1,1\dots 1)$ and $\vec{\Psi}=(\Psi(\alpha+\beta{x_1},\lambda)\dots \Psi(\alpha+\beta{x_N},\lambda))^T$.
And $$\Psi(\alpha+\beta{x_i},\lambda)=\log\bigg\{\sum_{z\in\Omega}\exp\big\{\big(\vec{\alpha}^T+\vec{\beta}^Tx_i\big)z_i+\mathrm{Tr}(Z\Lambda Z^T)\big\}\bigg\}$$ Either way the last term looks very nasty, especially the summation over all possible $z$. In order to find ML estimates $\hat{\theta}=(\hat{\vec{\alpha}}, \hat{\vec{\beta}}, \hat{\vec{\lambda}})$ of $\theta=(\vec{\alpha}, \vec{\beta}, \vec{\lambda})$ one has to compute the derivatives with the respect to $\theta$, set the obtained equations to zero and solve them. So the obtained system will look like: $$\begin{cases} \frac{\partial \ l}{\partial \vec{\alpha}}=\vec{1}^TZ- \vec{1}^T\frac{\partial \vec{\Psi}}{\partial \vec{\alpha}}=0\\ \frac{\partial \ l}{\partial \vec{\beta}}=\vec{x}^TZ- \vec{1}^T\frac{\partial \vec{\Psi}}{\partial \vec{\beta}}=0\\ \frac{\partial \ l}{\partial \Lambda}=Z^TZ- \frac{\partial \ \mathrm{Tr}(\vec{1}\!\otimes\!\vec{\Psi})}{\partial \Lambda}=0 \end{cases}$$ here $\otimes$ - is the outer product. So the solution of those equations will yield you ML estimates. Do I understand you correctly that $\Omega=(0,1)$?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.