# Hadamard Product Matrix Norm Inequality

Let $$A \odot B$$ denote the Hadamard or entry-wise product of two matrices with equivalent dimensions. In this post ( Hadamard product: Optimal bound on operator norm ) it is claimed without proof that if $$A$$ is positive-definite, then $$\|A \odot B\|_2 \leq \max_{i, j}|A_{i,j}| \|B\|_2$$ where $$\| \cdot \|_2$$ is the matrix operator norm induced from the Euclidean norm. Does anyone know how to prove this or have a reference? To me it does not seem so simple.

See (3.7.9) of Charles R. Johnson, Matrix Theory and Applications, American Mathematical Society, 1990, p.113. Essentially, for any two square matrices $$A$$ and $$B$$, if $$A=X^\ast Y$$ for some (probably rectangular) matrices $$X$$ and $$Y$$, then $$\pmatrix{\|B\|_2(I\circ X^\ast X)&A\circ B\\ (A\circ B)^\ast&\|B\|_2(I\circ Y^\ast Y)} =\pmatrix{X^\ast X&A\\ A^\ast&Y^\ast Y}\circ\pmatrix{\|B\|_2I&B\\ B^\ast&\|B\|_2I}$$ is positive semidefinite, by Schur product theorem. Consequently, $$A\circ B=\|B\|_2(I\circ X^\ast X)^{1/2}\,C\,(I\circ Y^\ast Y)^{1/2}$$ for some matrix $$C$$ with $$\|C\|_2\le1$$ and in turn, $$\|A\circ B\|_2\le\sqrt{\|I\circ(X^\ast X)\|_2\|I\circ(Y^\ast Y)\|_2}\,\|B\|_2.$$ In your case, as $$A$$ is positive definite, if you put $$X=Y=A^{1/2}$$, you will obtain $$\|A\circ B\|_2\le\|I\circ A\|_2\|B\|_2=\max_ia_{ii}\|B\|_2=\max_{i,j}|a_{ij}|\|B\|_2.$$
• Thank you. I understand everything except the line immediately after "Consequently". Why does $A \circ B = \|B\|_2 (I \circ X^* X)^{1/2}C (I \circ Y^* Y)^{1/2}$
• @BenB Let $D:=\|B\|_2(I\circ X^\ast X)$ and $\Lambda:=\|B\|_2(I\circ Y^\ast Y)$. If the Schur product on the LHS of the first displayed equation is positive semidefinite, the column space of $A\circ B$ must reside in the column space of $D$ and the row space of $A\circ B$ must reside in the row space of $\Lambda$. Hence $A\circ B=DC'\Lambda$ for some matrix $C'$. By absorbing $D^{1/2}$ and $\Lambda^{1/2}$ into $C'$, we get $A\circ B=D^{1/2}C\Lambda^{1/2}$ for some matrix $C$ that has the same column space as $D$'s and the same row space as $\Lambda$'s. Commented Jun 16, 2019 at 3:14
• In the special case where $D$ and $\Lambda$ are positive definite, you may simply take $C=D^{-1/2}(A\circ B)\Lambda^{-1/2}$. In the previous comment, $D^{1/2}$ and $\Lambda^{1/2}$ are possibly singular and we are essentially taking inverses of $D^{1/2}$ and $\Lambda^{1/2}$ on some subspaces. Commented Jun 16, 2019 at 3:25
• I see how the proof carries through in the case that $A$ is positive-definite since that implies $D$ and $\Lambda$ will be positive definite and then you can take the inverses like you said and it works out nicely. Then by perturbation and continuity the positive-definite case implies the case where $A$ is positive semi-definite and I am happy. Although I still can't see why if $A$ is not positive definite why $A \circ B$ is necessarily in the column space of $D$ or the row space of $\Lambda$.
• @BenB Suppose $(A\circ B)v$ does not lie inside the column space of $D$. Then its orthogonal projection $u$ onto $\ker(D)$ will be nonzero. It follows that when $k$ is a sufficiently large positive real number, \begin{aligned} &\pmatrix{-ku^\ast&v^\ast}\pmatrix{D&A\circ B\\ (A\circ B)^\ast&\Lambda}\pmatrix{-ku\\ v}\\ &=k^2u^\ast Du-2k\Re\left(u^\ast(A\circ B)v\right)+v^\ast\Lambda v\\ &=-2k\|u\|^2+v^\ast\Lambda v<0. \end{aligned} The case for $\Lambda$ is similar. Commented Jun 16, 2019 at 12:05