Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

1
vote
0answers
15 views

Gradient of the Lie exponential map on SO(n)

I am interested in computing the gradient of $f(e^A)$ when $A$ is a skew-symmetric matrix. If we write $e^A = B$ and we denote the gradient of the function $f$ on the ambient space evaluated at $B$ ...
-2
votes
0answers
16 views
1
vote
0answers
21 views

matrix block multiplication definition, properties and applications

I would like to have a clear definition of matrice block multiplication, its properties and some applications. If possible, some book references. Suppose we have $A B $ , where $A$ and $B $ are ...
1
vote
0answers
17 views

Finding the product of two normal distributions

Using the formulas given in section 8.1.8 of this handbook, what is going to be the result of following integral \begin{equation} \int\mathcal{N}_{\mathbf{A}\mathbf{X}}(\mathbf{m}_1,\Sigma_1)\mathcal{...
2
votes
0answers
89 views

Matrix functions reducing eigenspectrum degeneracy?

Standard matrix functions fulfill $spec(f(A))=f(spec(A))$ relation for eigenspectrum, maintaining degeneracy if $f$ is injection. For simplicity we can focus on real symmetric matrices. However, ...
1
vote
0answers
27 views

What can I say about $\mathbf{X}$ if $\mathbf{y}^{\mathrm{T}} \mathbf{X} \mathbf{y} = \mathbf{z}^{\mathrm{T}} \mathbf{X} \mathbf{z}$?

Consider the $N$-dimensional positive semidefinite matrix $\mathbf{X}$ and two $N$-dimensional vectors $\mathbf{y}$ and $\mathbf{z}$ with $\|\mathbf{y} \|^{2} = \| \mathbf{z} \|^{2}$. If I know that $\...
0
votes
1answer
56 views

Write the derivative of the lower-triangular matrix $L(t)$ in terms of $L(t)$, $L^{-1}(t)$ and $\frac{d}{dt}A(t)$, where $A(t)=L(t)L^T(t)$

Let $A(t)$ be a symmetric positive definite matrix, thus by Cholesky decomposition, we have $A(t)=L(t)L^T(t)$ where $L(t)$ is lower triangular. Suppose $A(t)$ is differentiable. I want to write $\...
0
votes
0answers
20 views

Derivative of Hadamard product with respect to matrix

I'm trying to calculate this derivative wrt matrix $F_{i}$ and simplify the whole expression: $ \frac{ \partial J_{term 1}}{\partial \mathbf{F_{i}}}= 2 (\sum_{j:G(i,j)=1} (\mathbf{W}_{i,j} \...
-1
votes
0answers
12 views

How to solve this matrix orthogonal problem? [on hold]

The problem is $Q^*=\arg\min \limits_{Q}Tr(Q^TAQ) +\frac{\mu}{2}||Q^TX-E ||_F^2\ \ s.t. Q^TQ=I$ ,where $Tr$ is matrix trace operator. Thanks.
1
vote
2answers
29 views

Why is $\triangledown_x x^TA^TAx = 2A^TAx$ not $2x^TA^TA$?

A proof I'm looking at shows $\triangledown_x x^TA^TAx = 2A^TAx$. I did Matlab symbolic calculation to verify this, but I found the converse. It should be $2x^TA^TA$. $\frac{d}{dx}x^TA^TAx = \\ [ ...
0
votes
1answer
33 views

How to determine basis and dimension of subspaces?

I'm new into linear algebra and I encountered the following problem: In $\mathbb{R}^4$ we conside the following subspaces: $U=\{(x.y.z.w)\in \mathbb{R}^4 | x+y+z=0\}$, $V=\mathscr{L}((1,1,0,0), (2,-...
1
vote
1answer
23 views

Calculate variations of eigenvectors w.r.t input matrix

From the eigen decomposition : $$A = PDP^T$$ I would like to calculate $dP$ w.r.t $dA$. I start like this : $$dA = dPDP^T + PdDP^T + PDdP^T$$ Since we only consider variations in eigenvectors and ...
0
votes
0answers
23 views

Question on matrix representation of equations

The question is: In a flock of wild Hoatzins, the females can be classified as being either chicks (up to 1 year old) or adults. Each year, for every 100 adult females, 50 female chicks are born. ...
0
votes
1answer
20 views

How to get 4x4 matrix from integrating column vector and its transpose?

This is from Finite Element Methods. Part of potential energy approach. Can someone show me how to come to this symetric matrix: $$ [K]=EI/L^3\begin{bmatrix} 12&6L&-12&6L\\ &4L^2&-...
1
vote
0answers
24 views

Minimization Problem involving PCA

Starting from this equation to estimate: $\DeclareMathOperator{\tr}{Trace}$ $$ X = F^k \Lambda' ^ {k} + e, $$ with $ X$ an $T\times N $ matrix, $ F$ is $T\times k $ and $ \Lambda$ is $k\times k $, ...
5
votes
1answer
45 views

Points of continuity in $ M_n (k) $ of minimal polynomials

The goal is to show that $\Gamma $ the set of points of continuity of $ M \mapsto \pi_M $ is $ \{ M \in M_n(k) , \chi_M = \pi_M \} $ . Where $ \pi_M $ is the minimal polynomial of $M$ and $ \chi_M $ ...
0
votes
2answers
28 views

It can happen that the norm 1 of a matrix and the infinite norm are different?

I have practiced some exercises with these two norms and in all of them I had the same result, until I tried with $\begin{pmatrix} 5 & -3 & 2 \\ 4 & 8 & -4 \\ 2 & 6 & -1 \\ \...
0
votes
1answer
44 views

What is the name of this type of matrix multiplication?

I don't know the name of this type of matrix multiplication. In this type, we multiply each same indexes of matrices and generate our new matrix. Please help me find out. Normal matrix multiplication ...
1
vote
1answer
16 views

Summation of polynomial matrix multiplication in terms of vector outer product

Consider the following summation $$ \sum_{i=1}^{T-1}C(A^i-A^{i-1})Bx_{t-i} $$ where $A$ is a $d \times d$ diagonal matrix, i.e. $A=\text{diag}(\alpha_1,\cdots,\alpha_d)$, $C$ is an $m \times d$, $B$ ...
-1
votes
0answers
17 views

Aligning point cloud using camera local coordinates only [closed]

I have the stanford bunny point clouds taken from different views and the config file containing all the local camera poses for each of these clouds. ...
2
votes
1answer
36 views

gradient and hessian of $e^{x^Tx}$

I want to find gradient and hessian of $e^{x^Tx}$ My attempt: $\nabla = 2x^Te^{x^Tx}$ Hessian $= 2e^{x^Tx}I + 2xx^Te^{x^Tx}$ Is that correct?
0
votes
1answer
33 views

How does the $Tr(nX^{n-1})$ become $n(X^{n-1})^T$?

For a unstructured square complex matrix X. Show that $\frac{\partial Tr(X^n)}{\partial X}=n(X^{n-1})^T $ and $\frac{\partial Tr(X^n)}{\partial X^*}=n(X^{n-1})^T$. I think $\frac{\partial Tr(X^n)}{...
0
votes
0answers
29 views

Gradient and hessian of $\log(1+x^TAx)$

I want to find Gradient and hessian of $log(1+x^TAx)$ where $A$ is positive semi definite. My attempt: $\nabla = x^T(A+A^T)/1+x^TAx$ Hessian = $(A+A^T)(1+x^TAx)-(x^T(A+A^T))^Tx^T(A+A^T)/(1+x^TAx)^2$ ...
2
votes
1answer
54 views

Why is $\frac{\partial \operatorname{Tr}(\mathbf X^n)}{\partial \mathbf X}=n\left(\mathbf X^{n-1}\right)^T ?$

Why is $\frac{\partial \operatorname{Tr}(\mathbf X^n)}{\partial \mathbf X}=n\left(\mathbf X^{n-1}\right)^T ?$ and why is $\frac{\partial (\ln[\det(\mathbf X)])}{\partial \mathbf X}=\mathbf X^{-T} ?$ ...
1
vote
0answers
23 views

Derivative of a scalar given by a function times its hessian

Is there any way of doing the below but by avoiding $d^3f\over dx^3$? If for a row vector $x$ e.g. size (3x1): $f=f(x)=scalar$ $g={df\over dx}$ i.e. the gradient $H={d^2f\over dx^2}$ i.e. the ...
1
vote
0answers
44 views

Continuity of matrix trace functionals

Let $P$ be a positive $n\times n$ matrix over $\mathbb{C}$, let $f:(0,\mathbb{R})\rightarrow\mathbb{R}$ be a continuous function, and define a function on positive definite $n\times n$ matrices as $$ ...
0
votes
2answers
36 views

Derivatives Across Summations

So, I took one intro course in Tensor calculus and this problem reminds of that, except I can't quite recall how derivatives work with respect to components, or what those derivatives produce. ...
3
votes
1answer
38 views

Eigenvalues of the Product of a Diagonal and a Symmetric Matrix

Let $A \in \mathbb{R}^{n\times n}$ be a symmetric matrix and $D \in \mathbb{R}^{n\times n}$ be a diagonal matrix with positive entries. Prove that the matrix $P:=DA$ has real eigenvalues.
2
votes
2answers
51 views

Derivative of a vector times its transpose

I am trying to work out how to solve a derivative of the form: $${d \over dx}(M(x)M(x)^T)$$ where M is a vector. In my case specifically, M is the (1x3) vector $$M(x)={df(x) \over dx}$$ where f(...
0
votes
1answer
44 views

Prove: $\nabla_A \text{tr}\left(ABA^TC\right) = CAB + C^TAB^T$

Edit: there's another post asking the same thing, but it is not satisfactorily answered. At least not in what I believe is close to the simplest way. Trying to prove property 3 below, in a "clean" ...
1
vote
1answer
30 views

Derivative of $\text{trace}(U^T x y^T V)$ with respect to $x$

I'm trying to compute the derivative of $\text{trace}(U^T x y^T V)$ with respect to $x$, where $U \in \mathbb{R}^{d_x \times k}$, $V \in \mathbb{R}^{d_y \times k}$, $x \in \mathbb{R}^{d_x}$, and $y \...
2
votes
1answer
22 views

Necessary conditions for convergence of matrix series [closed]

I am interested in necessary conditions under which the series $$\sum_{n=0}^\infty A^n \Omega (A^n)^{\top}$$ converges wrt. a submultiplicative matrix norm, where A is diagonalizable and $\Omega$ is ...
0
votes
1answer
41 views

Search for a constant Metric $\eta$ satisfying the a condition

I am looking for a $2 \times 2$ constant metric $\eta$, satisfying the following condition: $\eta A \eta^{-1}= A^T$. where $$A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$$ ...
4
votes
1answer
70 views

Differential of determinant in the Itô calculus

I'm trying to answer a question about a particular Itô equation $$\mathrm{d}X_t = AX_t\mathrm{d}t+X_tB(\mathrm{d}W_t)$$ where $X_t$ and $A$ are $n\times n$ square matrices and $B : M_n\to M_n$ ($M_n$ ...
3
votes
1answer
129 views

Directional derivatives of matrix trace functionals

Let $P$ be an $n\times n $ positive semidefinite matrix over $\mathbb{C}$, let $p\in\mathbb{R}$ be in the range $0<p<1$. Consider the function $g:[0,\infty)\rightarrow\mathbb{R}$ defined by $g(x)...
1
vote
0answers
29 views

Singularity of a block matrix

When is the matrix $\begin{bmatrix}A & B \\C & D \end{bmatrix}$ singular? I've found on the internet that if $A,B,C,$ or $D$ is non-singular and its Schur complements is non-singular then the ...
1
vote
2answers
50 views

Does this matrix identity hold? (Left and right-product with diagonal matrix)

I have been attempting to prove the following identity: $\frac{\partial [W^{\frac{1}{2}} K W^{\frac{1}{2}}]}{\partial \hat{f_i}} = K \frac{\partial W}{\partial \hat{f_i}}$ where $W^{\frac{1}{2}}$ is ...
-1
votes
1answer
37 views

Derivative of a products with matrix exponential

I should find the derivative with respect to the vector $a \in \mathbb{R}^{n}$ $\dfrac{\partial}{\partial a}(a^{T}exp(aa^{T})a)$ Answer should be in a matrix form I tried to decompose the ...
1
vote
1answer
65 views

Matrix differentiation: $\frac{\partial{w}}{\alpha}$ for $w=(X^\top X + \alpha \boldsymbol{I})^{-1}X^\top y$

What is $\frac{\partial{w}}{\partial{\alpha}}$ for $w=(X^\top X + \alpha \boldsymbol{I})^{-1} X^\top y$ where X is an $N \times D$ matrix, y is an N dimensional vector, $\boldsymbol{I}$ is an identity ...
0
votes
0answers
34 views

General chain rule - transpose

There is a function: $$g_{(x)}:\mathbb{R}^n \rightarrow \mathbb{R}$$ The $1_{st}$ derivative is gradient, $\mathbb{R}^{n}$ column vector. Second derivative is Hessian, $\mathbb{R}^{n*n}$ matrix. ...
2
votes
1answer
69 views

Trace of a Matrix: when to use? what is trace trick?

On calculating log-likelihood function for some multivariate distributions, such as multivariate Normal, I see some examples where the matrices are suddenly changed to trace, even when the matrix is ...
0
votes
1answer
27 views

Unsure of how to proceed through Trace Property

I'm having difficulty figuring out how to derive the following from Andrew Ng's CS229 lecture notes. $$\nabla_A \textrm{Tr } ABA^{T}C = CAB + C^TAB^T $$ where $\textrm{Tr }$ is the trace operator ...
0
votes
1answer
38 views

Simplify sum of quadratic forms to have matrix variable $\sum_i x_i^T A_i x_i$?

Let $A_i$ be a square symmetric matrix of size $n \times n$ and $x_i$ be the $i$-th row of matrix $X$ of size $m \times n$. Consider $f(x_1, x_2,...,x_m) = \sum_{i=1}^m x_i^T A_i x_i $. I need to ...
1
vote
0answers
28 views

Trace of the square root inside commutation property

Hi I'am looking forward to prove this statement, which seems to be true (checked numerically) even if I cannot find it anywhere and didn't manage to prove it : $\forall B\in M_{n,r}(\mathbb{R}),Tr(\...
1
vote
1answer
46 views

What are the roots in this function.

I am trying to find roots for a particular function. It has reduced to the following expression. $$ \frac{d f(\lambda)}{d \lambda}=8\lambda + 2\text{Trace}(Q\Sigma Q^\top)- \sum_i \frac{2 M_i}{1- 2\...
0
votes
1answer
33 views

Linear Matrix differential equation

Let $A,B$ be non-singular matrices of dimension $n\times n$. Is there a way to solve the differential equation $$ f(x)Bx=A\nabla_x f(x)? $$ I've looked in many places and it doesn't seem to be ...
1
vote
0answers
49 views

Upper bound for $(AB-BA)x$

Given matrices $A,B\in\mathbb{R}^{n\times n}$where matrix $A$ is a diagonal matrix and $B$ is an upper triangular matrix. I'm looking for an upper bound for the expression \begin{align*} (AB-BA), \end{...
0
votes
1answer
41 views

Derivative of multivariate normal density

I am not too familiar with matrix calculus, how can we go about taking the derivative of this quantity: $$ \nabla_{x}Ax\operatorname{det}(2\pi\boldsymbol\Sigma)^{-\frac{1}{2}} \, e^{ -\frac{1}{2}(\...
1
vote
3answers
63 views

Eigenvalues and eigenvectors of $4\times 4$ matrix

Let M the $4\times 4$ matrix: $M = \begin{pmatrix} 0 &-2 & 4 &-2 \\ 1 &1 &-2 &-1 \\ 0 &0 &0 &0 \\ 1 &-1 &2 &-3 \end{pmatrix} $ and $M^n = (...
2
votes
1answer
62 views

Is there a closed formula for the positive-definite matrix square root?

Let $S_{>0}$ be the space of symmetric positive definite real $n \times n$ matrices. Is there a closed-form formula for the positive square root function $\sqrt \cdot :S_{>0} \to S_{>0}$? ...