Show that scalar-valued function of a matrix is convex Consider the mapping
$$f(X) = g\left(\frac{b}{a^TXa}\right),$$
where $g$ is a convex function, $b$ is a strictly positive scalar, $a$ is a real vector, and $X$ is restricted to be symmetric and strictly positive definite (a covariance matrix).
An example for $g$ is $g(y) = y - \ln y -1$.
Are there conditions under which $f$ is convex (or quasi-convex) in $X$, such that gradient-based minimization can be used to find that $X$ which minimizes $f$?
(Also of interest is the minimzation of $\sum_i g(b_i/a^TXa)$.)
 A: If $g$ is convex and nondecreasing, which is the case for your example, then $f(X)$ is convex. This is because $b/(a^TXa)$ is convex on the domain of interest. (Since $X$ is positive definite, we can be sure that $a^TXa>0$.) The composition of a convex, nondecreasing function with a convex function is convex.
However, that does not mean you can use a simple gradient-based method to solve this. You must take mathematically sound steps to ensure that $X$ remains positive semidefinite during the minimization process. For instance, you can use a projected gradient approach, which alternates between two phases. In the first phase, a simple gradient step is taken to yield a new point $X_+$, which may be indefinite. In the second phase, a projection is performed, such as:
$$X_{++} = \min_Z \| X_+ - Z \|_F$$
which produces the "closest" positive definite result $X_{++}$. This projection is actually pretty easy to explain: just perform the Schur decomposition $X_{+}=Q\Lambda Q^T$, then construct a new $\Lambda_+$ satisfying $(\Lambda_+)_{ii}=\max\{\Lambda_{ii},0\}$; then $X_{++}=Q\Lambda_+Q^T$.
