4
$\begingroup$

Dear Linear Algebra experts,

According to Schur Product Theorem, if both $A \succ 0$ and $B \succ 0$ are positive definite, then the Hadamard product of $(A \circ B) \succ 0$ is also positive definite.

However, can Hadamard product $(A \circ B) \succ 0$ be positive definite, if $A \succeq 0$ (positive semi-definite) and $B \succ 0$ (positive definite)?

If yes, can you prove it? Thank you so much.

$\endgroup$

3 Answers 3

4
$\begingroup$

When $A\succeq0$ and $B\succ0$, the Hadamard product $A\circ B$ is positive definite if and only if $A$ has not any zero rows.

Proof 1. $A\circ B$ is positive semidefinite. It is positive definite if and only if it is non-singular. By Oppenheim's inequality, whenever $A$ and $B$ are positive semidefinite matrices, $\det(B)\prod_ia_{ii}\le\det(A\circ B)$. So, in your case, if $A\circ B$ is singular, some $a_{ii}$ must be zero and hence the $i$-th row of $A$ is zero. Conversely, if $A$ has a zero row, clearly $A\circ B$ is singular.

Proof 2. One of the usual proofs of Schur product theorem (such as the one on Wikipedia or the one in Horn and Johnson's Matrix Analysis) can be modified to prove our necessary and sufficient condition for the positive definiteness of $A\circ B$.

Suppose the positive semidefinite matrix $A$ has rank $k$. By spectral decomposition, we may write $A=\sum_{i=1}^kv_iv_i^\ast$ where $\{v_1,\ldots,v_k\}$ is a set of mutually orthogonal eigenvectors corresponding to the positive eigenvalues of $A$. Note that $$ x^\ast(A\circ B)x =\sum_{i=1}^k x^\ast\left((v_iv_i^\ast)\circ B\right)x =\sum_{i=1}^k x^\ast\left(\operatorname{diag}(v_i)B\operatorname{diag}(v_i^\ast)\right)x =\sum_{i=1}^k (x\circ\bar{v_i})^\ast B(x\circ\bar{v_i}). $$ Since $B$ is positive definite, $x^\ast(A\circ B)x=0$ if and only if $x\circ\bar{v}_i=0$ for every $i$. The latter statement is in turn equivalent to $\operatorname{diag}(x)\overline{V}=0$, or equivalently, $\operatorname{diag}(\bar{x})V=0$, where $V$ is the augmented matrix $[v_1|v_2|\cdots|v_k]$. It follows that $A\circ B$ is positive definite if and only if $V$ has not any zero row, i.e. if and only if $A$ has not any zero row.

$\endgroup$
12
  • $\begingroup$ thank you for your help and time $\endgroup$
    – user550103
    Apr 28, 2018 at 9:20
  • $\begingroup$ By the way, is there a typo in the eigen-decomposition of $A$, i.e., eigenvalues say $\lambda_i$ in $A = \sum_{i=1}^{k} \lambda_i v_i v_i^*$. And we assumed that $A$ is Hermitian to utilize such eigen-decomposition? $\endgroup$
    – user550103
    Apr 28, 2018 at 11:55
  • 1
    $\begingroup$ @user550103 It isn't a typo. Since $A$ is positive semidefinite, its eigenvalues are nonnegative and we may absorb $\sqrt{\lambda_i}$ into $v_i$ whenever $\lambda_i>0$. $\endgroup$
    – user1551
    Apr 28, 2018 at 12:39
  • 1
    $\begingroup$ @user550103 Yes, all positive definite matrices are Hermitian, at least in the context of linear algebra or Schur product theorem. $\endgroup$
    – user1551
    Apr 28, 2018 at 13:24
  • 1
    $\begingroup$ @user550103 A real matrix is called positive definite if it is symmetric and $x^TAx>0$ for every nonzero vector $x$. If $A$ is not symmetric, we do not speak of its definiteness. This, as I've said before, is how positive definite matrices are defined in linear algebra. In some disciplines (e.g. in engineering), positive definiteness is defined differently, but in linear algebra and in particular, in the context of Schur product theorem, $A$ is always assumed to be Hermitian. $\endgroup$
    – user1551
    May 6, 2018 at 17:22
3
$\begingroup$

Yes, for example take $A$ with all entries $1$.

A longer answer:

Note: the Gram matrix $(\langle v_i , v_j \rangle)$ of the system of vectors $v_i$ has rank equal to the rank of $(v_i)$.

If $A$ is the Gram matrix of the system $(v_i)$, and $B$ of the system $w_i$ then $A\circ B$ is the Gram matrix of the system $v_i \otimes w_i$

$$G(v_i) \circ G(w_i)= G(v_i\otimes w_i)$$

So the question is: what is the rank of the system $v_1\otimes w_1, \ldots ,v_n\otimes w_n$?

If $v_i$ has rank $1$ then rank of $v_i\otimes w_i$ $\le $ rank of $w_i$, with equality if all of the $v_i$ are $\ne 0$ (the answer above).

Consider the case $n=3$, and the systems of rank $2$ $$(e_1, e_2, -(e_1+e_2)), (e_1, e_2, -(e_1+e_2))$$ One checks right away that the rank of $$(e_1\otimes e_1, e_2\otimes e_2, (e_1+e_2)\otimes (e_1+e_2) )$$ is three. So we got a matrix $A=G(e_1, e_2, e_1+e_2)$ positive semidefinite such that $A\circ A$ is positive definite. Explicitely: $$A=\left(\begin{matrix}1&0&-1\\0&1&-1\\-1&-1&2\end{matrix}\right ), \ \ A\circ A =\left(\begin{matrix}1&0&1\\0&1&1\\1&1&4\end{matrix}\right )$$

$\bf{Added:}$ The above calculations work in general.

If $v_1$, $\ldots$, $v_{n-1}$ are linearly independent, $v_n$ dependent on $v_1$, $\ldots$, $v_{n-1}$ but not a multiple of any of them, then the Gram matrix $(\langle v_i, v_n\rangle)$ is psd of determinant $0$, but $(\langle v_i \otimes v_i, v_j \otimes v_j\rangle)$ is positive definite ($\det > 0$).

So we have lots of example of psd $A$ with $\det A=0$ such that the Hadamard square $A^{\circ 2}$ is positive definite

(here is a calculation for $3\times 3$ matrices of cosines that are degenerate and their Hadamard square-- generically $\det A^{\circ 2}>0$).

$\endgroup$
5
  • $\begingroup$ thank you for your help and time $\endgroup$
    – user550103
    Apr 28, 2018 at 9:20
  • $\begingroup$ I have a question on your first statement: "A with all entries 1". If I take $A = \begin{bmatrix} 1 &1 \\ 1 & 1 \end{bmatrix}$ then eigenvalues $\rho(A) = \left\{0, 2\right\}$, i.e., it is positive semi-definite. If I do Hadamard product say $A \circ A$, then the eigenvalues are same $\rho(A \circ A) = \left\{0, 2\right\}$, which means $A \circ A$ is positive semi-definite. Or Do I misunderstand something? $\endgroup$
    – user550103
    Apr 28, 2018 at 13:17
  • $\begingroup$ But your second example is as expected $\endgroup$
    – user550103
    Apr 28, 2018 at 13:19
  • 1
    $\begingroup$ @user550103: the first statement was meant : if $A$ has all entries $1$ then $A\circ B = B$ so will be positive for $B$ positive, even if $A$ has rank $1$. $\endgroup$
    – orangeskid
    Apr 28, 2018 at 13:27
  • $\begingroup$ thank you for the clarification $\endgroup$
    – user550103
    Apr 28, 2018 at 13:30
2
$\begingroup$

Let $M_n$ be the set of complex matrices of order $n$. Let $A\in M_n$ be a positive semidefinite matrix and $B\in M_n$ be a positive definite matrix.

Notice that if any column of $A$ is identically zero then $A\circ B$ can not be positive definite.

Thus, a necessary condition for positive definiteness of $A\circ B$ is no column of $A$ must be identically zero. Next, we prove the sufficiency of this condition.

In order to prove that, we must notice that $A\circ B$ is a submatrix of $A\otimes B\in M_{n^2}$, where $\otimes$ stands for the Kronecker product of these matrices.

Now, let $e_1,\ldots,e_n$ be the canonical basis of $\mathbb{C}^n$. Consider the vectors: $\{e_1\otimes e_1, \ldots, e_n\otimes e_n\}\subset C^{n^2}$

Let $P=\sum_{i=1}^ne_ie_i^t\otimes e_ie_i^t\in M_{n^2}$ be the orthogonal projection onto the subspace generated by $\{e_1\otimes e_1, \ldots, e_n\otimes e_n\}$. Define the Hermitian matrix $C=P(A\otimes B)P\in M_{n^2}$.

Notice that $\Im(C)\subset\text{span}\{e_1\otimes e_1, \ldots, e_n\otimes e_n\}$. Moreover, the matrix which represents the restriction of $C$ to $\text{span}\{e_1\otimes e_1, \ldots, e_n\otimes e_n\}$ on the basis $\{e_1\otimes e_1, \ldots, e_n\otimes e_n\}$ is $A\circ B$.

In order to prove that $A\circ B$ is invertible (assuming that no column of $A$ is identically zero), it is enough to show that $\ker(C)\cap\text{span}\{e_1\otimes e_1, \ldots, e_n\otimes e_n\}=\{0\}$.

If $C(\sum_{i=1}^na_ie_i\otimes e_i)=0$ then $\sum_{i=1}^na_iAe_i\otimes Be_i=0$. But $\{Be_1,\ldots,Be_n\}$ are the columns of B, which are linear independent, since $B$ is invertible. We can find $v_j\in\mathbb{C}^n$ such that $v_j^tBe_j=1$ and $v_j^tBe_i=0$ for every $j\neq i$.

So $0=(Id\otimes v_j^t)(0)=\sum_{i=1}^na_iAe_i\otimes v_j^tBe_i=a_jAe_j$.

Since $Ae_j$ is not zero (no column of $A$ is zero) then $a_j=0$, for every $j$.

Thus, $\ker(C)\cap\text{span}\{e_1\otimes e_1, \ldots, e_n\otimes e_n\}=\{0\}$ and $C$ restricted to $\text{span}\{e_1\otimes e_1, \ldots, e_n\otimes e_n\}=\{0\}$ is invertible. So $A\circ B$ is invertible.

We already know that $A\circ B$ is positive semidefinite.

$\endgroup$
1
  • $\begingroup$ thank you for your help and time $\endgroup$
    – user550103
    Apr 28, 2018 at 9:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .