0
$\begingroup$

I have a function with the form like this, $F(\beta) = ln (\int_{X\beta}^{\infty} \phi(u|0,\Sigma) du)$. Here $X, \Sigma$ are matrices. $\beta$ and $u$ are vectors. $\phi$ is the multivariate normal density function.

I need to get the second derivative of this function.

I'm poor in doing matrix calculus. But, I derive the result for the first derivative as follows,

$\frac{\partial F(\beta)}{\partial \beta} = \frac{1}{\int_{X\beta}^{\infty} \phi(u|0,\Sigma) du} \frac{\partial \phi(u|0,\Sigma)}{\partial \beta} =\frac{1}{\int_{X\beta}^{\infty} \phi(u|0,\Sigma) du} \frac{\partial \sqrt{(2\pi)^k |\Sigma|}exp(-\frac{1}{2}(X\beta)^T\Sigma^{-1}X\beta)}{\partial \beta}= \frac{1}{\int_{X\beta}^{\infty} \phi(u|0,\Sigma) du} \sqrt{(2\pi)^k |\Sigma|}exp(-\frac{1}{2}(X\beta)^T\Sigma^{-1}X\beta) \frac{\partial -\frac{1}{2} (X\beta)^T\Sigma^{-1}X\beta}{\partial \beta}= \frac{1}{\int_{X\beta}^{\infty} \phi(u|0,\Sigma) du} \sqrt{(2\pi)^k |\Sigma|}exp(-\frac{1}{2}(X\beta)^T\Sigma^{-1}X\beta) (-\frac{1}{2}X^T\Sigma^{-1}X\beta-\frac{1}{2}(X\beta)^T\Sigma^{-1}X)$

Is my first derivative correct? Can anyone show me the result for the second derivative?

$\endgroup$
0
$\begingroup$

Restating the problem $$\eqalign{ y &= X\beta \cr \phi(y) &= \phi_o\,\exp(-\tfrac{y^T\Sigma^{-1}y}{2}) \cr p = \exp(F) &= \int_y^\infty\phi(u)\,du \cr }$$ Since the variable $y$ only appears in the lower limit of integration, and not explicitly in the kernel inside the integral, Leibniz's rule is simply $$\eqalign{ dp &= -\phi(y)\,dy = -\phi(y)\,X\,d\beta \cr }$$ We can also calculate the differential of $p$ directly in terms of $F$, with the help of the Hadamard $(\odot)$ product
$$\eqalign{ dp &= p\odot dF = {\rm diag}(p)\,dF = P\,dF \cr }$$ Equating the two expression allows us to find the gradient wrt $\beta$ $$\eqalign{ dF &= -\phi(y)\,P^{-1}X\,d\beta\cr \frac{\partial F}{\partial\beta} &= -\phi(X\beta)\,P^{-1}X \cr &= -\phi_o\,\exp\Big(\tfrac{-\beta^TX^T\Sigma^{-1}X\beta}{2}\Big)\,{\rm diag}\big(\exp(F)\big)^{-1}X \cr\cr }$$ So that's the gradient; the Hessian will be much more complicated. Are you sure you need it?

$\endgroup$
  • $\begingroup$ thanks. Hessian is exactly what I need. This function is actually part of a multinational probit likelihood function. I need this to calculate the variance of $\beta$. I didn't understand why $p=diag(p)$ in Equation 5. Can you enlighten my on that? Is the Hessian the following: $\frac{\partial^2 F}{\partial\beta^2} = -\phi_o\,\exp\Big(\tfrac{-\beta^TX^T\Sigma^{-1}X\beta}{2}\Big)\,(\beta^T(X^T\Sigma^{-1}X+X^T(\Sigma^{-1})^TX)){\rm diag}\big(\exp(F)\big)^{-1}X + -\phi_o\,\exp\Big(\tfrac{-\beta^TX^T\Sigma^{-1}X\beta}{2}\Big)\,{\rm diag}\big(\exp(F)\big)^{-2}\phi(X\beta)XX$ $\endgroup$ – Ding Li Jun 8 '18 at 4:35
  • $\begingroup$ @DingLi Since $(F,\beta)$ are vectors, the gradient is already a matrix, and the Hessian will be a $3^{rd}$ order tensor. You cannot write it using standard matrix/vector notation, you must use index notation and work it out element by element. $\endgroup$ – greg Jun 8 '18 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.