6
$\begingroup$

Let $A \odot B$ denote the Hadamard or entry-wise product of two matrices with equivalent dimensions. In this post ( Hadamard product: Optimal bound on operator norm ) it is claimed without proof that if $A$ is positive-definite, then $$\|A \odot B\|_2 \leq \max_{i, j}|A_{i,j}| \|B\|_2$$ where $\| \cdot \|_2$ is the matrix operator norm induced from the Euclidean norm. Does anyone know how to prove this or have a reference? To me it does not seem so simple.

$\endgroup$
0

1 Answer 1

3
$\begingroup$

See (3.7.9) of Charles R. Johnson, Matrix Theory and Applications, American Mathematical Society, 1990, p.113. Essentially, for any two square matrices $A$ and $B$, if $A=X^\ast Y$ for some (probably rectangular) matrices $X$ and $Y$, then $$ \pmatrix{\|B\|_2(I\circ X^\ast X)&A\circ B\\ (A\circ B)^\ast&\|B\|_2(I\circ Y^\ast Y)} =\pmatrix{X^\ast X&A\\ A^\ast&Y^\ast Y}\circ\pmatrix{\|B\|_2I&B\\ B^\ast&\|B\|_2I} $$ is positive semidefinite, by Schur product theorem. Consequently, $$ A\circ B=\|B\|_2(I\circ X^\ast X)^{1/2}\,C\,(I\circ Y^\ast Y)^{1/2} $$ for some matrix $C$ with $\|C\|_2\le1$ and in turn, $$ \|A\circ B\|_2\le\sqrt{\|I\circ(X^\ast X)\|_2\|I\circ(Y^\ast Y)\|_2}\,\|B\|_2. $$ In your case, as $A$ is positive definite, if you put $X=Y=A^{1/2}$, you will obtain $$ \|A\circ B\|_2\le\|I\circ A\|_2\|B\|_2=\max_ia_{ii}\|B\|_2=\max_{i,j}|a_{ij}|\|B\|_2. $$

$\endgroup$
7
  • $\begingroup$ Thank you. I understand everything except the line immediately after "Consequently". Why does $A \circ B = \|B\|_2 (I \circ X^* X)^{1/2}C (I \circ Y^* Y)^{1/2}$ $\endgroup$
    – BenB
    Commented Jun 15, 2019 at 22:54
  • $\begingroup$ @BenB Let $D:=\|B\|_2(I\circ X^\ast X)$ and $\Lambda:=\|B\|_2(I\circ Y^\ast Y)$. If the Schur product on the LHS of the first displayed equation is positive semidefinite, the column space of $A\circ B$ must reside in the column space of $D$ and the row space of $A\circ B$ must reside in the row space of $\Lambda$. Hence $A\circ B=DC'\Lambda$ for some matrix $C'$. By absorbing $D^{1/2}$ and $\Lambda^{1/2}$ into $C'$, we get $A\circ B=D^{1/2}C\Lambda^{1/2}$ for some matrix $C$ that has the same column space as $D$'s and the same row space as $\Lambda$'s. $\endgroup$
    – user1551
    Commented Jun 16, 2019 at 3:14
  • 1
    $\begingroup$ In the special case where $D$ and $\Lambda$ are positive definite, you may simply take $C=D^{-1/2}(A\circ B)\Lambda^{-1/2}$. In the previous comment, $D^{1/2}$ and $\Lambda^{1/2}$ are possibly singular and we are essentially taking inverses of $D^{1/2}$ and $\Lambda^{1/2}$ on some subspaces. $\endgroup$
    – user1551
    Commented Jun 16, 2019 at 3:25
  • $\begingroup$ I see how the proof carries through in the case that $A$ is positive-definite since that implies $D$ and $\Lambda$ will be positive definite and then you can take the inverses like you said and it works out nicely. Then by perturbation and continuity the positive-definite case implies the case where $A$ is positive semi-definite and I am happy. Although I still can't see why if $A$ is not positive definite why $A \circ B$ is necessarily in the column space of $D$ or the row space of $\Lambda$. $\endgroup$
    – BenB
    Commented Jun 16, 2019 at 9:54
  • $\begingroup$ @BenB Suppose $(A\circ B)v$ does not lie inside the column space of $D$. Then its orthogonal projection $u$ onto $\ker(D)$ will be nonzero. It follows that when $k$ is a sufficiently large positive real number, \begin{aligned} &\pmatrix{-ku^\ast&v^\ast}\pmatrix{D&A\circ B\\ (A\circ B)^\ast&\Lambda}\pmatrix{-ku\\ v}\\ &=k^2u^\ast Du-2k\Re\left(u^\ast(A\circ B)v\right)+v^\ast\Lambda v\\ &=-2k\|u\|^2+v^\ast\Lambda v<0. \end{aligned} The case for $\Lambda$ is similar. $\endgroup$
    – user1551
    Commented Jun 16, 2019 at 12:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .