0
$\begingroup$

I'm trying to show that the Hessian of the negative of the log likelihood with two parameters is positive definite, but I'm not sure how to go about it once I compute the Hessian.

The function is: $-log L(\alpha) =-\sum\limits_{i=1}^N ((1-y_i)(-\alpha_0 - \alpha_1x_i) - log(1 + e^{-\alpha_0 - \alpha_1x_i}))$

and I believe that the Hessian is:

\begin{bmatrix} \frac{e^{-\alpha_{0}-\alpha{1}x_i}}{(e^{-\alpha_{0}-\alpha{1}x_i}+1)^2}&\frac{e^{-\alpha_{0}-\alpha{1}x_i}x_i}{(e^{-\alpha_{0}-\alpha{1}x_i}+1)^2}\\\frac{e^{-\alpha_{0}-\alpha{1}x_i}x_i}{(e^{-\alpha_{0}-\alpha{1}x_i}+1)^2}&\frac{e^{-\alpha_{0}-\alpha{1}x_i}x_i^2}{(1+e^{-\alpha_{0}-\alpha{1}x_i})^2}\end{bmatrix}

How can I show that this matrix is positive definite for all $\alpha$ in order to prove convexity?

$\endgroup$
2
  • $\begingroup$ factor out the (nonnegative) denominator in front and look at the trace and determinant $\endgroup$
    – LinAlg
    Commented Oct 17, 2018 at 18:26
  • $\begingroup$ Hi, @LinAlg, I took your advice and got this: $det(H) = ac − b^2 = {e^{-\alpha_{0}-\alpha_{1}x_i}}{e^{-\alpha_{0}-\alpha_{1}x_i}x_i^2} - ({e^{-\alpha_{0}-\alpha_{1}x_i}x_i})^2 = e^{-2\alpha_{0}-2\alpha_{1}x_i}x^2 - e^{-2\alpha_{0}-2\alpha_{1}x_i}x^2 = 0$. Don't I need $det(H) > 0$ for it to be positive definite? $\endgroup$ Commented Oct 18, 2018 at 1:05

1 Answer 1

1
$\begingroup$

The Hessian simplifies to: $$\frac{e^{-\alpha_{0}-\alpha{1}x_i}}{(1+e^{-\alpha_{0}-\alpha{1}x_i})^2}\begin{bmatrix} 1 & x_i\\x_i & x_i^2\end{bmatrix}.$$ The factor is positive and does not affect positive (semi)definiteness. The matrix has trace $1+x_i^2$ and determinant $0$. Therefore, the eigenvalues are $0$ and $1+x_i^2$. This matrix is therefore positive semidefinite.

$\endgroup$
2
  • $\begingroup$ Ahh, I see. I got the same thing, just by factoring out less. However, since I'm trying to prove convexity, don't I need the matrix to be positive definite and not just semidefinite? $\endgroup$ Commented Oct 18, 2018 at 1:31
  • 1
    $\begingroup$ @MiguelBarrera No you do not. The function is convex. $\endgroup$
    – LinAlg
    Commented Oct 18, 2018 at 2:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .