4
$\begingroup$

I'm having a difficult time determining if the following function is convex

$$f(X) = \log {\rm det}(X^T A X)$$

where $A \in \mathbb{R}^{r \times r}$ is a symmetric positive definite matrix and $X \in \mathbb{R}^{r \times u}$ with $u < r$ and $X'X = I$. I've worked on find the second derivative with respect to $X$, but am not very confident in my solution. Any suggests on steps to proceed would be appreciated.

EDIT: Because the domain of $X$ is nonconvex, the function cannot be convex, as pointed out by @RobertIsrael.

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ Use $\det(XY) = \det X \det Y$, $\det X^T = \det X$ and $\det X > 0$ if $X$ is positive definite to reduce the problem to $g(\alpha) = \log \alpha$ where $\alpha \in R^{+}$. $\endgroup$ – Biswajit Banerjee Dec 4 '15 at 3:59
  • $\begingroup$ Is this true for all $X$ and $Y$? My impression was that this only held for square $X$ and $Y$. $\endgroup$ – user23658 Dec 4 '15 at 4:09
  • $\begingroup$ No only square. Need to think again. $\endgroup$ – Biswajit Banerjee Dec 4 '15 at 4:11
  • $\begingroup$ If its any help, $\frac{\partial}{\partial X} \log{\rm det}(X^TAX) = 2(X^TAX)^{-1}AX$. I was stuck on taking the second derivative. $\endgroup$ – user23658 Dec 4 '15 at 4:12
  • 5
    $\begingroup$ $X^T X = I$? The set of such $X$ is not convex, so how can you even ask if $f(X)$ is convex? $\endgroup$ – Robert Israel Dec 4 '15 at 4:55
2
$\begingroup$

(EDIT: If $A,X$ are both square, then ...) As the comment suggest, we know that since the determinant is multiplicative and $\det X^T=\det X$, we have that $$\det(X^TAX)=\alpha\det(X)^2$$ where $\alpha=\det(A)>0$.

However, the function $g(x)=x^2$ is not logarithmically convex, therefore the function $f(X)$ is not either.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Does anything change when you consider that $X$ and $A$ are not of the same dimension and $X$ is not square? $\endgroup$ – user23658 Dec 4 '15 at 4:07
  • $\begingroup$ @user23658, that is an interesting question. $\endgroup$ – charlestoncrabb Dec 4 '15 at 4:10
  • $\begingroup$ The multiplicity of the determinant only holds under square $X$ and $A$, I believe. Not sure tho! That's why I ask. $\endgroup$ – user23658 Dec 4 '15 at 4:10
  • $\begingroup$ My instinct is telling me the square case suggests the general $f(X)$ is not concave, let me think on it... $\endgroup$ – charlestoncrabb Dec 4 '15 at 4:12
0
$\begingroup$

The Hessian can be used to find a sufficient (but not necessary) condition for convexity. According to the author, the first derivative of the objective function is $$ D^1 := \frac{\partial}{\partial X} (\log\det(X^T A X)) = 2(X^T A X)^{-1} AX = 2 B^{-1} A X $$ The second derivative can be computed by going into index notation (summation over repeated indices). Then $$ (D^1)_{in} = 2B^{-1}_{il} A_{l m} X_{mn} $$ Then $$ (D^2)_{inpq} := \frac{\partial (D^1)_{in}}{\partial X_{pq}} = 2\left[\frac{\partial B^{-1}_{il}}{\partial X_{pq}} X_{mn} + B^{-1}_{il} \frac{\partial X_{mn}}{\partial X_{pq}}\right] A_{lm} $$ Now, (without symmetrizing) $$ \frac{\partial X_{mn}}{\partial X_{pq}} = \delta_{mp}\delta_{nq} $$ Therefore, $$ B_{il}^{-1} \frac{\partial X_{mn}}{\partial X_{pq}} A_{lm} = B_{il}^{-1} \delta_{mp}\delta_{nq} A_{lm} = \delta_{nq} B_{il}^{-1} A_{lp} $$ Consider the case where $A = I$. Then $B = X^T A X = X^T X = I$ and $D^1 = 2X$. So $$ (D^2)_{inpq} = 2\delta_{ip}\delta_{nq}\,. $$ That is the simplest result you can get because $X$ is not square. Not sure if this helps in determining convexity.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Not sure if you saw the edit above but the derivative you use should have an inverse and be scaled by 2. $\endgroup$ – user23658 Dec 4 '15 at 4:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.