4
$\begingroup$

I am sure the answer to this question is out there, but I cannot find it, maybe because I don't know the correct torm for 'matrix multiplication from left and right'.

Consider a matrix $W \in \mathbb{R}^{m \times n}$ and a square matrix $A \in \mathbb{R}^{m \times m}$. What can we say about the determinant $$\det (W^TAW)?$$

In the case that $W$ is also square, since the determinant then commutes, we have $$\det (W^TAW) = \det (A WW^T) = \det (A) \det (WW^T) = \det(A) \det(W^TW).$$

Do we have a similar result when $W$ is not square? We can not expect the two last equalities on the left generally hold, since one of $\det (WW^T)$ and $ \det(W^TW)$ will be equal to $0$, but maybe it holds for the respective one that might have full rank?

Edit: Maybe this works indeed, using singular value decomposition:

Let $W = U \Sigma V$ and assume $W$ has full rank, and assume that $m<n$, such that $U$ is an orthogonal $m\times m$ matrix, $V$ is an orthogonal $n \times n$ matrix, and $\Sigma \in \mathbb{R}^{m\times n}$ is of the form $\begin{pmatrix}D & 0\end{pmatrix}$ with $D$ diagonal.

Then $$\det (W^T AW) = \det (V^T \Sigma^T U^T A U \Sigma V) = \det ( \Sigma^T U^T A U \Sigma VV^T) = \det ( \Sigma^T U^T A U\Sigma)$$

Bot now if we denote $U^TAU=B$, then $$\Sigma^T B \Sigma = \begin{pmatrix}D \\ 0\end{pmatrix} B \begin{pmatrix}D & 0\end{pmatrix} = DBD $$. But then

$$\det(\Sigma^T U^T A U\Sigma) = \det (\Sigma^TB\Sigma) = \det (\Sigma^T\Sigma B) = \det(D^2) \det(B) = \det(D^2) \det(U^TAU) = \det(D^2) \det(A) =\det(WW^T)\det(A).$$

Is this correct? second edit: the 'solution' above must contain an error, since we obviously have that the determinant is zero in the case that $m<n$...

$\endgroup$
4
  • $\begingroup$ I'd do a small example, e.g. $1\times 2$ by $2\times 2$ by $2\times 1$, just to see if there is any chance for this to be true. $\endgroup$
    – hardmath
    Commented Aug 13, 2021 at 14:51
  • 2
    $\begingroup$ When $m>n$, Cauchy-Binet formula gives $\det(W^TAW)=\sum_{|I|=n}\sum_{|J|=n}\det(W(I,[n]))\det(A(I,J))\det(W(J,[n]))$ where the index sets $I$ and $J$ run over all subsets of $[m]=\{1,2,\ldots,m\}$ of cardinalities $n$. $\endgroup$
    – user1551
    Commented Aug 13, 2021 at 15:09
  • $\begingroup$ @user1551 so would you expect any more simplification or is that all one can say? $\endgroup$
    – a_student
    Commented Aug 13, 2021 at 15:32
  • $\begingroup$ @a_student It might be helpful to note that many terms in the above sum might be zero, especially if you incorporate SVD $\endgroup$ Commented Aug 13, 2021 at 15:49

2 Answers 2

4
$\begingroup$

An analysis similar to yours works for the case where $m>n$. Let $U \Sigma V^T$ be the SVD of $W$, where $\Sigma$ has the form $$ \Sigma = \pmatrix{D\\0}. $$ Denote $B = U^TAU$. Partition $B$ into $$ B = \pmatrix{B_{11} & B_{12}\\ B_{21} & B_{22}}, $$ where $B$ is square of size $n$. We have $$ \Sigma^TB\Sigma = \pmatrix{D & 0}\pmatrix{B_{11} & B_{12}\\ B_{21} & B_{22}} \pmatrix{D \\ 0} = DB_{11}D. $$ Thus, we have $$ \det(W^TAW) = \det(\Sigma^TB\Sigma) = \det(DB_{11}D) = \det(B_{11})\det(D)^2\\ = \det(B_{11}) \det(W^TW). $$ This isn't as "nice" as the formula for the case of $m = n$, but perhaps you will find it interesting nevertheless.

$\endgroup$
1
  • $\begingroup$ Yes this seems more reasonable than what I wrote. I will accept the other answer since it got a creative solution, hope thats okay $\endgroup$
    – a_student
    Commented Aug 13, 2021 at 21:11
3
$\begingroup$

I suggest to use the generalization of the Matrix determinant lemma: $$ \det\left(\mathbf{Q} + \mathbf{UAV}^\textsf{T}\right) = \det\left(\mathbf{A}^{-1} + \mathbf{V}^\textsf{T}\mathbf{Q}^{-1}\mathbf{U}\right)\det\left(\mathbf{A}\right)\det\left(\mathbf{Q}\right). $$ with $\mathbf{Q} = \varepsilon {\mathbf{I}}_n$, $\mathbf{V}^\textsf{T} = \mathbf{U}^\textsf{T} = \mathbf{W}\in \mathbb C^{m\times n}$: $$ \det\left(\varepsilon {\mathbf{I}}_n + \mathbf{W}^\textsf{T}\mathbf{AW}\right) = \det\left(\mathbf{A}^{-1} + \varepsilon^{-1} \mathbf{W}\mathbf{W}^\textsf{T}\right)\det\left(\mathbf{A}\right)\det\left(\mathbf{\varepsilon {\mathbf{I}}_n}\right). $$ Note that here the determinants are calculated for matrices with different sizes. Having this in mind we write $$ \det\left(\varepsilon {\mathbf{I}}_n + \mathbf{W}^\textsf{T}\mathbf{AW}\right) = \varepsilon^{n-m} \det\left(\varepsilon \mathbf{A}^{-1} + \mathbf{W}\mathbf{W}^\textsf{T}\right)\det\left(\mathbf{A}\right). $$

Now let us tend $\varepsilon\to 0$. When $n>m$ we obtain $$ \det\left(\mathbf{W}^\textsf{T}\mathbf{AW}\right) = 0. $$ When $n=m$ we obtain $$ \det\left(\mathbf{W}^\textsf{T}\mathbf{AW}\right) = \det\left( \mathbf{W}\mathbf{W}^\textsf{T}\right)\det\left(\mathbf{A}\right). $$ When $m>n$ we obtain $$ \det\left( \mathbf{W}^\textsf{T}\mathbf{AW}\right) = \det\left(\mathbf{A}\right) \lim_{\varepsilon\to0} \frac{ \det\left(\varepsilon \mathbf{A}^{-1} + \mathbf{W}\mathbf{W}^\textsf{T}\right) }{\varepsilon^{m-n}}. $$ I do not know if this expression is somehow helpful but we can use it to obtain a closed-form expression in the particular case of $m=n+1$ by differentiating the determinant using Jacobi formula: $$ \det\left( \mathbf{W}^\textsf{T}\mathbf{AW}\right) = \det\left(\mathbf{A}\right) {\rm tr}[ {\rm adj}(\mathbf{W}\mathbf{W}^\textsf{T}) \mathbf{A}^{-1} ] ={\rm tr}[ {\rm adj}(\mathbf{W}\mathbf{W}^\textsf{T}) \cdot {\rm adj}\mathbf{A} ] . $$

$\endgroup$
2
  • $\begingroup$ interesting approach, how exactly do you yield the last line by the Jacobi formula? $\endgroup$
    – a_student
    Commented Aug 13, 2021 at 18:02
  • $\begingroup$ Since both the numerator and demoninator vanish as $\varepsilon\to0$, I used L'Hôpital's rule and differentiated the numerator and the denominator with respect to $\varepsilon$. The derivative of the denominator is ${\rm d}\varepsilon^{n+1-n}/ {\rm d}\varepsilon = 1$ and the derivative of the determinant in the numerator was calculated using Jacobi formula: ${\rm tr} ({\rm adj} (\varepsilon\mathbf{A}^{-1} + \mathbf{W}\mathbf{W}^{\rm T})\cdot \mathbf{A}^{-1})$, which tends to ${\rm tr} ({\rm adj} (\mathbf{W}\mathbf{W}^{\rm T})\cdot \mathbf{A}^{-1})$ as $\varepsilon\to0$ $\endgroup$
    – bcp
    Commented Aug 13, 2021 at 18:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .