Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Filter by
Sorted by
Tagged with
0 votes
1 answer
37 views

why this equality involving matrix holds true?

I am studying Lemma 11 of the paper I am having difficulty understanding on the last step, in particular, I have two questions: The first question (1) On the first equality of the last step on page 15,...
chloe's user avatar
  • 304
-1 votes
0 answers
20 views

Can anyone help with this matrix problem? [duplicate]

enter image description here In this image, there's a question about asking to get the determinant of matrix D. I'm stuck on how to represent it as using only a and n.
ryan lee's user avatar
0 votes
0 answers
24 views

Derivative with respect to a rectangular matrix

Consider the following; $$ J = L^{\text{T}}KR $$ where $$ L \in \mathbb{R}^{p}, ~~ K \in \mathbb{R}^{p \times m}, ~~ r \in \mathbb{R}^{m} $$ I am attempting to find $ \frac{dJ}{dK}$. This is what I ...
AdamsK's user avatar
  • 177
0 votes
1 answer
52 views

What is the definition for the dot product of a co-vector and a dyadic product of vectors?

The definition of the dot product of a second order tensor with a vector is: $$ (\mathbf{a} \otimes \mathbf{b}) \cdotp \mathbf{x} = (\mathbf{b} \cdotp \mathbf{x})\mathbf{a} $$ The definition of the ...
Tomek Dobrzynski's user avatar
-1 votes
0 answers
58 views

Derivative of $\mathrm{Tr}({\boldsymbol \Omega}\boldsymbol H(\mathrm{diag}(\boldsymbol u))^H\boldsymbol H(\mathrm{diag}(\boldsymbol u)))$

Noting $H\in\mathbb{C}^{M\times M}$ with the following form of: \begin{equation} \boldsymbol H=(\boldsymbol h+\boldsymbol g\mathrm{diag}(\boldsymbol u)\boldsymbol G)^{T}(\boldsymbol h+\boldsymbol g\...
parouersi's user avatar
0 votes
0 answers
67 views

Minimizer of two matrix energies

I am looking for a solution $X \in \mathbb{R}^{n \times n}$ which minimizes the following energy: $$ \textrm{min}_X \| X A X^T - B\|^2 + \mu \|X \Lambda X^T - \Lambda \|^2 $$ where $A, B \in \mathbb{...
tommym's user avatar
  • 317
2 votes
1 answer
59 views

When is von Neumann entropy differentiable?

Given a smoothly time-dependent density matrix (positive-semidefinite matrix with trace 1) $\rho(t)$, its von Neumann entropy is defined as $$ S(t)=\mathrm{Tr}(f(\rho(t))),\ f:[0,\infty)\ni x\mapsto -...
tuna's user avatar
  • 21
-1 votes
0 answers
23 views

Conditions on and b for LU facotorization (3 x 4 matrix) [closed]

enter image description here Hi,I am facing issues doing this problem. Any help would be appreciated:)
Levi Ackerman's user avatar
-2 votes
0 answers
38 views

what are max and min number of free variables in the solutions for a 5x4 matrix [closed]

if A is a 5 x 4 matrix that represents a homogeneous system of linear equations and A is not the zero matrix. what's the maximum number of free variables in the solution to the system? And what's the ...
Sara Ra's user avatar
0 votes
0 answers
89 views

How to Compute the Second Derivative of a Quadratic Maximization Problem with Respect to the Weight Parameter?

I am working on a problem involving the maximization of a function subject to a normalization constraint, specifically in the context of quadratic forms and eigenvalues. The objective function is ...
Krishnendu K's user avatar
0 votes
3 answers
58 views

How to show this matrix limit?

Let $A$ be a $m\times n(m<n)$ matrix, $\mu>0$. $A$ is a full rank matrix. I am interested in showing that $$\lim_{\mu\to 0} (A^TA+\mu I)^{-1}A^T= A^T(AA^T)^{-1},$$ where $I$ is the identity ...
Du Xin's user avatar
  • 35
-1 votes
0 answers
76 views

Find all X which satisfy an equation [closed]

What are the matrices X with complex entries for which exists A, invertible, with complex entries such that: $$AA'X + XA'A = O_n$$, where $A'$ denotes the transpose matrix of A? I think that the only ...
Bogdan Sprincenatu's user avatar
1 vote
2 answers
105 views

Element-to-Element Expression of a Matrix Exponential?

Given a matrix $B = exp(A)$, how do we express $B_{ml}$ using $A_{ij}$? If $A$ is a diagonal matrix, this is very straightforward: $B_{ii} = exp(A_{ii})$. Can we generalize it to a real square matrix $...
lostintimespace's user avatar
0 votes
0 answers
109 views

Backpropagation: Chain Rule for Matrix Exponential?

Recent linear state-space model papers like Mamba often use matrix exponential to discretize the system. They initialize the system in a continuous-time regime, and discretize it to run it like a ...
lostintimespace's user avatar
1 vote
3 answers
143 views

Closed form solution to matrix equation

I'm trying to solve the following matrix equation for $L$: $$A\cdot L + L^T = 0$$ where A is non-singular and both $A,L \in \mathbb{R}^{n\times n}$. I'm wondering if this could have a closed-form ...
supernova's user avatar
1 vote
2 answers
108 views

Does the derivative with respect to a matrix have a Kronecker product matrix representation?

I'm confused why I end up with two matrices that are transposes of each other when I take a tensor inner product of a third order tensor with a vector, when I use two different Kronecker product ...
Tomek Dobrzynski's user avatar
4 votes
1 answer
491 views

A matrix manipulation from Berezin's paper

I am reading a paper by Berezin, entitled General Concept of Quantization. He writes: and then: This ought to be some clever manipulation of matrices and yet I am not able to show how (1.4) follows ...
Atom's user avatar
  • 3,791
1 vote
1 answer
58 views

Derivative of a Euclidean distance matrix

Let's say I have $X \in \mathbb{R}^{n \times d}$, a collection of $n$ row vectors of size $d$. We can calculate an $n \times n$ distance matrix, $D$, from $X$, where each $\{i,j\}$ entry denotes the ...
Mercury's user avatar
  • 184
1 vote
3 answers
71 views

Confusion with matrix derivatives and component-wise gradients

According to this source (The Matrix Cookbook), we have that for $f(X) = \mathbf{a}^T X \mathbf{b}$ where $X \in \mathbb{R}^{n \times m}, \mathbf{a} \in \mathbb{R}^n, \mathbf{b} \in \mathbb{R}^m$, the ...
user9740643's user avatar
2 votes
1 answer
56 views

Derivative of trace of antisymmetrical Matrix

it is easy to show that the derivative of the trace $Tr(A)$ with respect to $A$ if A is a N x N matrix, is $\frac{\partial Tr(A)}{\partial A} = I_{N \times N}$ However, if I now assume A to be ...
Chgad's user avatar
  • 121
0 votes
1 answer
46 views

How to prove $SUS=\frac{(S|S)}{2}U$ with $S$ and $U$ are skew-hermitian with following conditions?

We denote the $(A|B):=-{\rm tr}(AB)$ as the inner product of the skew-hermitian matrix, which is easy to prove by the definition. I'm stuck by the problem when I read the Lie algebra of matrix group, ...
Krystal Justin's user avatar
0 votes
0 answers
26 views

Lower bound on a term involving positive definite matrix

If $A$ is a positive definite matrix, can we derive a lower bound for the term $x^T A y$, where $x, y$ are two vectors of the following form: $$(x-z)^T A (x-y) \geq \alpha (x-z)^T (x-y), $$ where $\...
Ki Chao's user avatar
  • 43
0 votes
2 answers
128 views

Using Matrix Calculus in Backpropagation derivation. Rules of order of matmul and transposition when taking derivatives in different layouts.

I define a neural network with $L$ layers ($L-1$ hidden layers). The forward pass is as follows: $$ \mathbf{a}^{(l)} = f(\mathbf{W}^{(l)}\mathbf{a}^{(l-1)}+\mathbf{b}^{(l)}) $$ Where $l \in [0,L]$ and ...
xTom's user avatar
  • 45
0 votes
1 answer
58 views

What kind of product operator appears in the chain rule involving a derivative wrt a matrix?

Let $$y = Ax$$ $$z = y^Ty$$. Per the chain rule, $$\frac{\partial{z}}{\partial{A}} = \frac{\partial{y}}{\partial{A}} \frac{\partial{z}}{\partial{y}} $$ Since, $$\frac{\partial{z}}{\partial{A}} = \frac{...
Tomek Dobrzynski's user avatar
2 votes
1 answer
93 views

how to calculate $e^{tA}$ with $tr(A)=0$ and invertible matrix $A\in{M_{2}(\mathbb{R})}$

How to calculate $e^{tA}$ with $tr(A)=0$ and invertible matrix $A\in{M_{2}(\mathbb{R})}$? By Cayley-Hamilton, we can get $$A^2+|A|I=0$$ Then we calculate the non-zero eigenvalues $$\left\{ \begin{...
Krystal Justin's user avatar
3 votes
2 answers
105 views

"Correct" way of seeing that $\frac{\partial}{\partial A} f(AB) = \frac{\partial f(X)}{\partial X} B^T$, where $X = AB$

Lemma: Let $A \in \mathbb{R}^{m\times n}$ and $B \in \mathbb{R}^{n\times k}$ be matrices, and let $f:\mathbb{R}^{m\times k} \to\mathbb{R}$ be a differentiable function. Let $X = AB$. Then $$ \frac{\...
Sebastian Monnet's user avatar
0 votes
1 answer
32 views

Vector by Matrix Derivatives in Computational Graphs

I'm trying to learn how to implement the back propagation pass given a computational graph. I've been following the following guide: https://homepages.inf.ed.ac.uk/htang2/mlg2022/tutorial-3.pdf In ...
David Yue's user avatar
  • 225
1 vote
1 answer
78 views

How can I minimize a quadratic matrix-valued function?

Consider the following optimization problem $$ \min_X \quad XAX^T + XB + B^TX^T \tag{1} $$ where $X \in \mathbb R^{m \times n}, A \succ 0,$ and $B \in \mathbb R^{n \times m}$. To precisely define a ...
mhdadk's user avatar
  • 1,393
0 votes
1 answer
58 views

Matrix Derivatives for MLE estimates

I'm reading Tipping and Bishop's paper on Probabilistic Pricipal Component Analysis. To get the MLE estimate (variables $W \in \Re^{d \times q}$ and $\sigma^2 \in \Re^+$). $N, d , q \in \Re$ and $S \...
Harry's user avatar
  • 21
0 votes
0 answers
93 views

Intuition for vector calculus

In my statistics class, I was introduced to Fisher Information. As it comes from the Taylor Expansion in vector form, I wanted to know terms were ordered in a certain way - whether it was just to make ...
Jackanap3s's user avatar
0 votes
1 answer
61 views

gradient of $\operatorname{tr}((AX)^2)$ w.r.t

For Hermitian matrices $A$ and $X$, I'm wondering how to calculate $\frac{\partial \operatorname{tr}(AXAX)}{\partial X}$. My attempt: Let $f(X) \mapsto \operatorname{tr}(AXAX)$. Expanding $f(X+H) = \...
user1239110's user avatar
0 votes
1 answer
15 views

Express the matrix expression $p(i,j)= (D I^\top) \cdot p$

Consider: a vector $D$ with dimension $n\times 1$ containing one element equal to one, and all the other elements equal to 0 a vector $I$ with dimension $1\times n$ containing one element equal to ...
Star's user avatar
  • 250
-1 votes
1 answer
57 views

Derivative of a trace of a partitioned matrix with respect to a sub matrix

I am trying to find the critical points of an optimization problem by taking the derivatives of a complicated function. One of the terms in the objective function involves taking the derivative of a ...
shani's user avatar
  • 777
0 votes
0 answers
28 views

Computing derivative of logarithm of a specific matrix function.

I am trying to compute equation $(5.23)$ from here where I replaced the variable $\hat{f}$ by $x$ for ease of notation. By definition $B(x) := I + W(x)^{1/2}KW(x)^{1/2}$ with $K$ and $W$ symmetric ...
edamondo's user avatar
  • 1,217
1 vote
2 answers
122 views

Adjugate of rank $n-1$ matrix

I'm working on this problem: Given $A$ be an $n\times n$ complex matrix with $\operatorname{rank} A = n-1$, how to find its adjugate matrix? I have solved the case when (algebraic) multiplicity of ...
Evan_Bradley's user avatar
0 votes
1 answer
99 views

derivative of $(Z\alpha-Y\alpha X\alpha)^2$ with respect to $\alpha$

I want to calculate the derivative of this function and solve the gradient equation $$f(\alpha)=(Z\alpha-Y\alpha X\alpha)^2$$ Where $Z,Y,X \in \mathbb{R}^n$ and $\alpha^T\in \mathbb{R}^n$. Hence all ...
domath's user avatar
  • 1,295
0 votes
0 answers
49 views

Simple matrix calculus, and yet I am struggling to understand

Here is my problem: We have $\mathbf{D} \in \Re^{m n}$, $\mathbf{W} \in \Re^{m q}$, and $\mathbf{X} \in \Re^{q n}$. Furthermore, $\mathbf{D} = \mathbf{W}\mathbf{X}$. (NOT an element wise ...
wrek's user avatar
  • 485
3 votes
4 answers
177 views

derivative of a matrix to a power of 1/2

I'm trying to solve the following matrix derivative : $$\frac{d}{dx}(I + x\Sigma)^{1/2},$$ where $I$ is identity matrix and $\Sigma$ is a constant (positive definite) matrix which is not a function of ...
jason 1's user avatar
  • 685
2 votes
1 answer
40 views

Step in derivation of Hadamard's First Variational Formula for Hermitian Matrices

This arose in trying to understand the details of a proof of Hadamard's first variational formula in Terence Tao's "Topics in Random Matrix Theory". Suppose $A$ is an $n \times n$ Hermitian ...
J.B.R's user avatar
  • 117
0 votes
1 answer
58 views

Find $X$ such that $ \left\| A \odot X - B \right\|_F^2$ is minimized

Find $X$ such that $ \left\| A \odot X - B \right\|_F^2$ is minimized, where $\odot$ denotes elementwise multiplication (or Hadamard product). My solution is as follows as: Let $f(X) = \left\| A \...
learning's user avatar
  • 651
1 vote
2 answers
97 views

Derivative of multiplied vectors and matrix with respect to single variable in vector

Let v be a vector $v = (a + bc_1, ..., a + bc_n)$. Thus, we can say $v = a1 + bc$, where $1 = (1,..., 1)$ and $c = (c_1,...c_n)$. Let $M$ be a symmetric and quadratic matrix of proportions $n * n$, ...
Mr Entscheidung's user avatar
0 votes
1 answer
22 views

Partial Derivative of Matrix Summation with respect to a component.

I am trying to take the partial derivative of the following expression with respect to the component $[A]_{i, j}$, where both matrices $A$ and $B$ are $d\times d$, and $B$ is a diagonal matrix (this ...
ab19775's user avatar
0 votes
3 answers
80 views

On the Hessian of the Log-Determinant and the solution provided in Stephen Boyd's textbook

This is a follow up to another question on the second-order approximation to log-determinant in Boyd's textbook, the excerpt can be found here: Here, $f$ is the log-determinant, $f(Z) = \log(\det(Z))$...
Olórin's user avatar
  • 5,393
1 vote
1 answer
54 views

Gradient of Function of Complex Matrices

Consider the following function: $$ f(T) = \| T^{T}TB - C\|^2_2 $$ where $T, B, $ and $C$ are all complex matrices. Let $T = X + iY.$ I wish to compute $\nabla f$ i.e. $\dfrac{\partial f}{\partial T}$....
Shreyas Bharadwaj's user avatar
1 vote
1 answer
37 views

Is there any method that can optimize the problem whose regularizer is kurtosis term?

I recently worked on an optimization problem, whose regularizer $g(x)$ is kurtosis. The overall optimization formula is as follows. $$\begin{align} \arg \min_x \frac12 \Vert Ax-b\Vert_2^2 + \lambda g(...
Leung Joe's user avatar
0 votes
1 answer
40 views

Let $T_k$ be a sequence of $n\times n$ positive matrices such that

Let $\{T_k\}$ be a sequence of $n\times n$ positive matrices such that $$\lim\limits_{N\to\infty}\frac{1}{N}\sum\limits_{k=1}^N T_k=0.$$ I want to see whether this ensures existence of a sub-sequence $...
DeltaEpsilon's user avatar
  • 1,086
1 vote
1 answer
95 views

Derivative of real-valued function that takes a matrix

I want to compute the partial derivative of a real-valued function that takes matrices as argmuents. The function has the form $$F(x,y,z) = ||g(x) \odot (S \cdot y) - z||,$$ where $x, y, z, S \in \...
lfische's user avatar
  • 11
0 votes
0 answers
58 views

Power series expansion of matrix logarithms

How does one derive the power series expansion for matrix logarithm? Let's suppose we define the matrix exponential in the standard way as a power series. I'm only interested in matrices so the most ...
maxical's user avatar
  • 579
2 votes
1 answer
84 views

Derivative of determinant of a matrix w.r.t a scalar

So as per the matrix cookbook, $\frac{d |\textbf{A}|}{d \alpha} = |\textbf{A}| \operatorname{Tr}(\textbf{A}^{-1} \frac{d \textbf{A}}{d \alpha})$ where $\textbf{A}$ is a matrix and $\alpha$ is a scalar....
sage76's user avatar
  • 41
3 votes
1 answer
65 views

How can I use Lagrangian Multipliers to maximize a General Rayleigh Quotient for Linear Discriminant Analysis

I'm reading up on Linear Discriminant Analysis and have hit the point where one wants to maximize: $$ \frac{v^{T}S_{b}v}{v^{T}S_{w}v} $$ with respect to $v$, where $S_b$ is the between class variance ...
Steven's user avatar
  • 33

1
2 3 4 5
77