4
$\begingroup$

I'm having a difficult time determining if the following function is convex

$$f(X) = \log {\rm det}(X^T A X)$$

where $A \in \mathbb{R}^{r \times r}$ is a symmetric positive definite matrix and $X \in \mathbb{R}^{r \times u}$ with $u < r$ and $X'X = I$. I've worked on find the second derivative with respect to $X$, but am not very confident in my solution. Any suggests on steps to proceed would be appreciated.

EDIT: Because the domain of $X$ is nonconvex, the function cannot be convex, as pointed out by @RobertIsrael.

$\endgroup$
12
  • $\begingroup$ Use $\det(XY) = \det X \det Y$, $\det X^T = \det X$ and $\det X > 0$ if $X$ is positive definite to reduce the problem to $g(\alpha) = \log \alpha$ where $\alpha \in R^{+}$. $\endgroup$ Dec 4, 2015 at 3:59
  • $\begingroup$ Is this true for all $X$ and $Y$? My impression was that this only held for square $X$ and $Y$. $\endgroup$
    – user23658
    Dec 4, 2015 at 4:09
  • $\begingroup$ No only square. Need to think again. $\endgroup$ Dec 4, 2015 at 4:11
  • $\begingroup$ If its any help, $\frac{\partial}{\partial X} \log{\rm det}(X^TAX) = 2(X^TAX)^{-1}AX$. I was stuck on taking the second derivative. $\endgroup$
    – user23658
    Dec 4, 2015 at 4:12
  • 5
    $\begingroup$ $X^T X = I$? The set of such $X$ is not convex, so how can you even ask if $f(X)$ is convex? $\endgroup$ Dec 4, 2015 at 4:55

2 Answers 2

2
$\begingroup$

(EDIT: If $A,X$ are both square, then ...) As the comment suggest, we know that since the determinant is multiplicative and $\det X^T=\det X$, we have that $$\det(X^TAX)=\alpha\det(X)^2$$ where $\alpha=\det(A)>0$.

However, the function $g(x)=x^2$ is not logarithmically convex, therefore the function $f(X)$ is not either.

$\endgroup$
4
  • $\begingroup$ Does anything change when you consider that $X$ and $A$ are not of the same dimension and $X$ is not square? $\endgroup$
    – user23658
    Dec 4, 2015 at 4:07
  • $\begingroup$ @user23658, that is an interesting question. $\endgroup$ Dec 4, 2015 at 4:10
  • $\begingroup$ The multiplicity of the determinant only holds under square $X$ and $A$, I believe. Not sure tho! That's why I ask. $\endgroup$
    – user23658
    Dec 4, 2015 at 4:10
  • $\begingroup$ My instinct is telling me the square case suggests the general $f(X)$ is not concave, let me think on it... $\endgroup$ Dec 4, 2015 at 4:12
0
$\begingroup$

The Hessian can be used to find a sufficient (but not necessary) condition for convexity. According to the author, the first derivative of the objective function is $$ D^1 := \frac{\partial}{\partial X} (\log\det(X^T A X)) = 2(X^T A X)^{-1} AX = 2 B^{-1} A X $$ The second derivative can be computed by going into index notation (summation over repeated indices). Then $$ (D^1)_{in} = 2B^{-1}_{il} A_{l m} X_{mn} $$ Then $$ (D^2)_{inpq} := \frac{\partial (D^1)_{in}}{\partial X_{pq}} = 2\left[\frac{\partial B^{-1}_{il}}{\partial X_{pq}} X_{mn} + B^{-1}_{il} \frac{\partial X_{mn}}{\partial X_{pq}}\right] A_{lm} $$ Now, (without symmetrizing) $$ \frac{\partial X_{mn}}{\partial X_{pq}} = \delta_{mp}\delta_{nq} $$ Therefore, $$ B_{il}^{-1} \frac{\partial X_{mn}}{\partial X_{pq}} A_{lm} = B_{il}^{-1} \delta_{mp}\delta_{nq} A_{lm} = \delta_{nq} B_{il}^{-1} A_{lp} $$ Consider the case where $A = I$. Then $B = X^T A X = X^T X = I$ and $D^1 = 2X$. So $$ (D^2)_{inpq} = 2\delta_{ip}\delta_{nq}\,. $$ That is the simplest result you can get because $X$ is not square. Not sure if this helps in determining convexity.

$\endgroup$
1
  • $\begingroup$ Not sure if you saw the edit above but the derivative you use should have an inverse and be scaled by 2. $\endgroup$
    – user23658
    Dec 4, 2015 at 4:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.