# Convexity of norm function

Suppose $A,B$ are two $n \times n$ matrices with real entries. Are there some simple conditions on $A,B$ under which the function $$x \mapsto \sqrt{\|Ax\|\|Bx\|}$$ from $\Bbb{R}^n \to \Bbb{R}$ is convex?

I use some numerical algorithm to calculate an optimum, and a condition for the convergence of the algorithm is the convexity of the given function. The algorithm converges for all functions of this form, but I did not manage to prove the convexity of the above function.

-
If $A=diag(1,0,0,\ldots,0)$ and $B=diag(0,1,0,\ldots,0$ then $f(x)=\sqrt{|x_1|}\sqrt{|x_2|}$ which is not convex. Have you some assumptions about the matrices on which you are working? – Davide Giraudo Apr 30 '12 at 10:54
@DavideGiraudo: My matrices are of the form $\begin{pmatrix}a\cos \theta & a \sin\theta \\ -b\sin \theta & b\cos \theta \end{pmatrix}$, but matrices in Didier's counterexample below (and your's also) are of this form. – Beni Bogosel Apr 30 '12 at 11:01

Here are some bad news: assume the dimension is $n=2$, fix some positive $s$, assume that $A$ is the diagonal matrix with diagonal $(1,s)$ and $B$ is the diagonal matrix with diagonal $(s,1)$, and that $x$ is the first basis vector and $y$ is the second basis vector. Let $u:z\mapsto\sqrt{\|Az\|\cdot\|Bz\|}$.
Then $\|Ax\|=\|By\|=1$, $\|Ay\|=\|Bx\|=s$ and $\|A(x+y)\|=\|B(x+y)\|=\sqrt{1+s^2}$. Thus, $u(x)=u(y)=\sqrt{s}$ and $u(x+y)=\sqrt{1+s^2}$, hence $u(x+y)\gt u(x)+u(y)$ for every $s$ small enough. Since $u(tz)=tu(z)$ for every positive $t$ and every $z$, $u$ is not convex.
This example applies to every dimension $n\geqslant2$. It yields invertible matrices $A$ and $B$ if one wants them to be (choose $s\ne0$). To save the result you are interested in, one might have to restrict both matrices $A$ and $B$ to have a small condition number.