Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a problem. From wikipedia http://en.wikipedia.org/wiki/Positive-definite_matrix any function can be written as $$z^TMz$$ where z is a column vector and M is a symmetric real matrix. However this quadratic function is strictly convex only when M is symmetric positive definite. Why ?, I thought any quadatic function should be convex ? doesn't $$z^TMz$$ >0 shows only that the range of this function is greater than zero? 1. Why isn't any symmetric matrix M(which represents a quadratic function) convex ?

  1. Why is it only the case when $$z^TMz$$ denotes convex ?

Thanks

share|improve this question
1  
Suppose $M=\pmatrix{1&0\cr0&-1\cr}$. Then $M$ is symmetric and real, and $z^tMz=x^2-y^2$. Is $x^2-y^2$ convex? What's your definition of convex? –  Gerry Myerson Oct 10 '12 at 1:17
    
For function to be convex I mean: en.wikipedia.org/wiki/Convex_function $$f(tx_1+(1-t)x_2) <tf(x_1) +(1-t)f(x_2) $$ for any 0 < t < 1, x1 != x2. –  Jing Oct 10 '12 at 1:24
    
OK, so, does $f=x^2-y^2$ fit that definition? What if you pick points with $f(x_1)=f(x_2)=0$, say? –  Gerry Myerson Oct 10 '12 at 1:30
    
hmm... I am confused, can you give me some sketch of mathematical proof? f(x) is convex iff M(f(x)) is SGD ? thankx –  Jing Oct 10 '12 at 1:42
    
First, have you found an example to show that for the $M$ I gave, the function is not convex? Second, what is SGD? –  Gerry Myerson Oct 10 '12 at 1:59

1 Answer 1

up vote 3 down vote accepted

UPDATE: As pointed out in the comments by @Erik, positive definiteness is a sufficient condition for strict convexity. However in these case, positive definiteness is indeed directly implied since the second derivative is a positive definitive matrix.

PREVIOUS ANSWER: For any twice differentiable function, it is strictly convex if and only if, the Hessian matrix is positive definite. You can find it from any standard textbook on convex optimization. Now here the function at hand is $z^TMz$ which is clearly twice differentiable (by virtue of being quadratic). Now the Hessian of this function is $M$ (please verify yourself, it helped me a lot to memorize it). So $M$ should be positive definite for that quadratic function to be convex.

share|improve this answer
    
Thank you ! that's very clear. –  Jing Oct 10 '12 at 7:04
1  
The Hessian matrix does not need to be positive definite for a strictly convex function. Take the function $f(x) = \sum_{i=1}^Nx_i^4$, where $x = (x_1,\ldots,x_N)$; this has a Hessian matrix of $H(x) = 12\cdot\text{diag}(x_1^2,\ldots,x_N^2)$ which is only positive semi-definite (substitute $x=0$ to see this). Positive definiteness of the Hessian is only a sufficient condition for strict convexity, not a necessary one. A function is strongly convex if and only if its Hessian is positive definite. Your statement is true for quadratic forms, however. –  Erik Miehling Aug 30 at 3:47
1  
I completely agree with your comment. Has updated the answer. –  dineshdileep Aug 31 at 13:52

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.