16
$\begingroup$

Hi I would like to know whether the trace of the inverse of a symmetric positive definite matrix $\mathrm{trace}(S^{-1})$ is convex.

Actually I know that the trace of a symmetric positive definite matrix $S\in M_{m,m}$ is convex since we can find $B\in M_{n,m}$ such that $S=B^T\times B$ then we can write the trace as the sum of scalar quadratic forms, i.e. $\mathrm{trace}(S)=\mathrm{trace}(B^T\times B)=\sum_{j=1}^mb_j^T\times b_j$ where $b_j$ is the $j^{th}$ column of $B$.

for instance if we have $trace([\begin{array}{cc} 1 & 2 \\ 3 & 4 \\ \end{array}] \times [\begin{array}{cc} 1 & 3 \\ 2 & 4 \\ \end{array}])= [\begin{array}{cc} 1 & 2 \\ \end{array}]\times [\begin{array}{c} 1 \\ 2 \\ \end{array}]+ [\begin{array}{cc} 3 & 4 \\ \end{array}]\times [\begin{array}{c} 3 \\ 4 \\ \end{array}]=30$

And so I wonder if $\mathrm{trace}(S^{-1})$ is convex too..

$\endgroup$
4
  • $\begingroup$ So you are asking if the function $S\longmapsto \mbox{trace}\;S^{-1}$ is convex on symmetric positive definite matrices? $\endgroup$
    – Julien
    Feb 8, 2013 at 0:48
  • $\begingroup$ yes I would like to know if that function is convex $\endgroup$
    – user2987
    Feb 8, 2013 at 0:51
  • 2
    $\begingroup$ The fact that the trace of the matrix itself is convex is obvious, because the trace is linear. That stuff about $B^T B$ is irrelevant (and wrong, since you want to look at a function of $S$, not of $B$). $\endgroup$
    – Robert Israel
    Feb 8, 2013 at 1:12
  • $\begingroup$ yes I want to look at the function itself but remember that a function can be convex just in a specific interval. And in this case I guess that this interval is the symmetric positive definite matrices.. $\endgroup$
    – user2987
    Feb 9, 2013 at 22:24

2 Answers 2

Reset to default
29
$\begingroup$

Yes, it is. Consider $S(t) = A + t B$ where $A$ is symmetric positive definite and $B$ is symmetric. It is enough to show that $$\left.\dfrac{d^2}{d t^2} \text{Tr}(S(t)^{-1})\right|_{t=0} \ge 0$$ Now $$ S(t)^{-1} = (A (I + t A^{-1} B))^{-1} = A^{-1} - t A^{-1} B A^{-1} + t^2 A^{-1} B A^{-1} B A^{-1} + \ldots$$ so $$ \left. \dfrac{d^2}{\partial t^2} \text{Tr}(S(t)^{-1}) \right|_{t=0} = 2 \text{Tr}(A^{-1} B A^{-1} B A^{-1})$$ But $A^{-1} B A^{-1} B A^{-1} = C A^{-1} C^T$ where $C = A^{-1} B$ and $A^{-1}$ is positive definite, so $C A^{-1} C^T$ is positive semidefinite, and therefore $\text{Tr}(CA^{-1} C^T) \ge 0$.

$\endgroup$
11
  • 1
    $\begingroup$ Very nice Robert, +1. $\endgroup$
    – Julien
    Feb 8, 2013 at 1:50
  • $\begingroup$ So can we found the scalar quadratic form for $\mathrm{trace}(S^{-1})$ like I did for $\mathrm{trace}(S)$? $\endgroup$
    – user2987
    Feb 11, 2013 at 21:17
  • $\begingroup$ What scalar quadratic form? $\endgroup$
    – Robert Israel
    Feb 11, 2013 at 21:21
  • $\begingroup$ like the one that I have written for $\mathrm{trace}(S)=\mathrm{trace}(B^T\times B)=\sum_{j=1}^mb_j^T\times b_j$ where $b_j$ is the $j^{th}$ column of $B$. $b_j^T\times b_j$ is scalar quadratic form for $Q=I_{m\times m}$. $\endgroup$
    – user2987
    Feb 12, 2013 at 0:55
  • 6
    $\begingroup$ A twice differentiable function $f$ on an open subset $U$ of a vector space $V$ is convex iff $$ \left.\dfrac{d^2}{dt^2} f(u + tv)\right|_{t=0} \ge 0$$ for all $u \in U$, $v \in V$. The point is that other values of $t$ (for which $u+tv \in U$) correspond to other choices for $u$ at $t=0$. $\endgroup$
    – Robert Israel
    Nov 19, 2013 at 4:31
2
$\begingroup$

We can prove that $Tr\{X^{-1}\}$ is convex function by considering an arbitrary line, $X = Z + tV$, where $V \in S^n$ and $Z \in S^n_{++}$. Let $g(t) = f(Z+tV)$ for all $t \in \mathbb R$ such that $Z+tV \in S^n_{++}$. If (for all $X \in S^n_{++}$ and all $V \in S_n$) $g$ is convex then $f$ is convex. Consider \begin{align} g(t) &= Tr\{X^{-1}\} = Tr\{(Z+tV)^{-1}\}\\ & = Tr\{Z^{-1/2}(I+tZ^{-1/2}VZ^{-1/2})^{-1})Z^{-1/2}\} \\ & = Tr\{Z^{-1}(I+tZ^{-1/2}VZ^{-1/2})^{-1})\} \qquad \qquad \because \; Tr\{BA\} = Tr\{AB\} \\ \end{align} Since, $(Z^{-1/2}VZ^{-1/2})^T = Z^{-1/2}VZ^{-1/2}$ as $Z,V \in S^n$. We can write, $ Z^{-1/2}VZ^{-1/2} = Q\Sigma Q^T$, where $Q$ is orthogonal matrix and $\Sigma$ is a diagonal matrix with eigen values $\lambda_i$'s $ Z^{-1/2}VZ^{-1/2}$ of as diagonal entries. Hence,

\begin{align} g(t) & = Tr\{Z^{-1}(I+tQ\Sigma Q^T)\} \quad \because \; Tr\{BA\} = Tr\{AB\} \\ & = Tr\{Z^{-1}Q^T(I+t\Sigma)Q\} \quad \because \; Q^{-1} = Q^T \\ & = Tr\{QZ^{-1}Q^T(I+t\Sigma)\} \\ & = \sum_{i=1}^n (QZ^{-1}Q^T)_{ii} \frac{1}{1+t\lambda_i}\\ \implies g''(t) &= \sum_{i=1}^n (QZ^{-1}Q^T)_{ii} \frac{2\lambda_i^2}{(1+t\lambda_i)^3} \end{align} As, $(1+t\lambda_i)>0$ and $QZ^{-1}Q^T$ is symmetric and positive definite matrix ($\because Z^{-1} \in S^n_{++} $) resulting in the diagonal entries of $QZ^{-1}Q^T$ being positive values. Hence $g''(t)>0$ is always true. This concludes that $f$ is convex.

$\endgroup$
2
  • $\begingroup$ You went a bit too quickly when assuming $\lambda_i> 0$. Indeed, the $\lambda_i$ are the eigenvalues of the matrix $A=Z^{-1/2}VZ^{-1/2}$, which is symmetric because $Z^{-1/2}$ and $V$ are, but which is not necessarily positive definite because condition $Z+tV\succ 0$ does not necessarily require $V\succ 0$.However, once can easily show that $I+tA$ is positive definite, which implies $1+t\lambda_i>0$. We can then conclude by noticing that the function $g_i:t\mapsto \frac{1}{1+t\lambda_i}$ is convex on the interval $\{t \in \mathbb R : 1+t\lambda_i>0\}$. $\endgroup$
    – DodoDuQuercy
    Dec 3, 2021 at 15:26
  • 1
    $\begingroup$ I have made the changes. Hope it is fine now. Thanks @DodoDuQuercy $\endgroup$
    – Jayanth S
    Dec 4, 2021 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.