Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Filter by
Sorted by
Tagged with
1 vote
0 answers
34 views

Determine the Hessian of this function?

Given the function below, I would like to determine the Hessian with respect to $\mathbf{x}$, which will result in a $2 \times 2$ matrix. Note: $n$ and $\mathbf{z}$ are constants with respect to $\...
  • 452
2 votes
1 answer
22 views

Continuity of maximum eigenvalue function

Let $f:S^n_{+}(\mathbb{R})\rightarrow \mathbb{R}_+$ be defined as $$f(A)=\lambda_{\max}(A),$$ where $S^n_{+}(\mathbb{R})$ is set of all positive definite $n\times n$ matrices. What can we say about ...
  • 175
0 votes
1 answer
28 views

Derivative of a vector x wrt itself

I am attempting to better understand matrix calculus. Given a vector $x$, I understand that: $\frac{\mathrm{d} x^t}{\mathrm{d} x}= I$ but struggle to think about $\frac{\mathrm{d} x}{\mathrm{d} x}$. ...
  • 27
-1 votes
0 answers
42 views

How to compute gradient of a function of matrix? [closed]

Write the explicit formula of the gradient of $$ E(u) = \sum_{i=2}^{n-1}\sum_{j=2}^{m-1} \sin\left(\left(u[i+1, j] - u[i,j-1]\right)^2 + \epsilon\right) $$ with respect to the variable $u$, which is ...
  • 1
0 votes
0 answers
8 views

Vector by tt-Matrix multiplication [closed]

Let's say: Y is a matrix with size of [1,M] X is a matrix with size of [1,N] Z is a matrix with size of [N,M] It is needed to perform the product of X and Z in order to generate Y values (e.g., Y = ...
  • 1
1 vote
1 answer
25 views

Stuck in matrix manipulation and rearrangement

We have that $A$ is a symmetric matrix $n\times n$, $b$ is a $n\times 1$ vector, $C$ is a scalar, and $d$ is a $n\times 1$ vector. The expression $\left(Ad\right)'A^{-1}\left(Ad\right)-2b'A^{-1}Ad+C$ ...
  • 47
-4 votes
0 answers
22 views

How do I take derivative of a Matrix [closed]

I have a Matrix A and every element in A is a function of t or zero. A = [[2*t^2, 0], [0, t+5]] for this 22 matrix, how can I take the derivative of it?
0 votes
0 answers
29 views

How to solve AX=B for A when A, X and B are (2x2) matrices?

I've just been trying to solve this problem but I'm struggling, I've looked online and all the examples solve for X which seems more simple. What I'm trying to do is calculate a calibration matrix for ...
  • 1
0 votes
1 answer
17 views

Exponential matrix derivative to find the Hessian matrix of negative binomial regression

I am looking for the hessian matrix of the log likelihood function of negative binomial regression $$l\left( \cdot \right) =\sum ^{n}_{i=1}y_{i}\ln \left( \dfrac{\alpha \exp \left( x_{i}^{T}\beta \...
0 votes
1 answer
37 views

Gradient of $ \left( \mathbf{w}^T \mathbf{x} \right)^2 - 2\mathbf{w}^T \mathbf{x}$ w.r.t $\mathbf{w}$

That's it. So far I've tried the following but I'm not certain if I'm allowed to do that. \begin{align} \frac{d\left( \left( \mathbf{w}^T \mathbf{x} \right)^2 - 2\mathbf{w}^T \mathbf{x} \right)}{d\...
3 votes
0 answers
94 views

Show that $\det(xA+yB+zI_{n})=\det(yA+xB+zI_{n})$

We have $A$ and $B$ $(n×n)$ matrices with complex entries. We know that $A-B=AB-BA$. Show that $$\det(xA+yB+zI_{n})=\det(yA+xB+zI_{n})$$ for every $x,y,z$ complex numbers with $x+y≠0$. We can see that ...
2 votes
0 answers
41 views

What's the geometric meaning of total positivity?

Is there a geometric meaning of total positivity? I wanted to thought of it as a measure of volume but there wasn't a derivative definition associated with the total positivity. But does it define ...
1 vote
0 answers
37 views

Response function in weakly coupled differential equations

I have the following system of differential equations: \begin{equation} \frac{d}{dt} \, \begin{pmatrix} x_1 (t) \\ x_2 (t) \\ x_3(t) \\ x_4(t) \\ \vdots \\ x_M (t) \end{pmatrix} = \begin{pmatrix} f_1(...
  • 386
0 votes
0 answers
32 views

The differential of trace function

This answer states that the differential of a trace function is given by $d\text{Tr}(f(X)) = f'(X^T):dX$ where the colon $A:B$ means $\text{Tr}(A^TB)$. How do we prove it? Does any know any reference/...
0 votes
0 answers
37 views

Matrix derivatives for ML

Assume that our data is distributed according to a $\underline d$ dimensional multivariate Gaussian with $\bar \mu$ mean and $\Sigma$ covariance matrix: $$(\mathbf x_1, \dots, \mathbf x_n) \sim \...
0 votes
0 answers
53 views

Remove a vector from a subspace

Suppose I have a $N \times N$ matrix $A$ which is a projection matrix on the column space of another matrix $B=[b_1, \ldots, b_n], n<N $, that is, $A=BB^\dagger$, with $^\dagger$ denoting the ...
  • 745
7 votes
2 answers
210 views

Taylor series of a matrix exponential

I am looking to minimize the value of: $$g(t)=\mathrm{Tr}\left[\exp(X+tY)\right]$$ where both $X$ and $Y$ are symmetrical matrices with real coefficients. In general, $X$ and $Y$ do not commute so $\...
  • 1,835
0 votes
0 answers
15 views

PCA proof, trace move

In the PCA proof, I see this move: What is the "trace" algebra/rule that allow this trace-move?
  • 103
1 vote
1 answer
98 views

Deriving backpropagation equations - vectorization (regression)

I have a huge problem trying to derive the backpropagation equations. All the solutions I've found online are not detailed as I'd like, hence I'm here asking your help. First of all sorry for this ...
  • 111
0 votes
0 answers
47 views
+50

Functional derivative of $u \mapsto R\left(\int_{\Theta} \phi(\theta) \; \text{d}u(\theta)\right) + \lambda u(\Theta)$ for matrix-valued measure $u$

Let $\Theta$ be a closed connected manifold, $H$ and $F$ Hilbert spaces and $R \colon H \to \mathbb R$ and $\phi \colon \Theta \to F$ smooth. What is the functional derivative of $$f(u) := R\left(\...
  • 4,339
0 votes
1 answer
43 views

Definition of Derivative of Matrix

Let's assume A is $n\times 1$ constants, $X$ is $n\times 1$ vector. Does derivative of transpose(A)* X on X should be transpose(A) instead of A? I saw both transpose(A) and A from different resources ...
  • 31
1 vote
2 answers
26 views

Is the derivative of $\pmb x^\top (I+GB)^{-1} (I+GB)^{-\top}\pmb x$ with respect to $B$ a 4th-order tensor?

Is the derivative of $\pmb x^\top (I+GB)^{-\top} (I+GB)^{-1}\pmb x$ with respect to $B$ a 4th-order tensor? Where $\pmb x$ is a vector, and $B$ is a matrix. I followed the procedures in What is the ...
5 votes
2 answers
132 views

Matrix derivative of $f(X^T Y)$ w.r.t X

Given $X\in \mathbb{R}^{m\times n}$, $Y\in \mathbb{R}^{m\times k}$, $X^\top Y=Z\in \mathbb{R}^{n\times k}$, and $f:\mathbb{R}^{n\times k} \to \mathbb{R}$, we have the following: \begin{equation} f(X^\...
0 votes
2 answers
36 views

Hessian Matrix of Matrix Product

I am not sure how I can compute the Hessian Matrix of a trace of the matrix such as this: $$ f(w) := \operatorname{tr} \left( B w w^T A \right) $$ where $A$ and $B$ are $n \times n$ square matrices ...
0 votes
1 answer
35 views

Matrix exponential of the sum of a diagonal matrix and a rank-$1$ matrix

Suppose we have the matrix $D + u v^T$, where $D$ is a diagonal matrix and both $u$ and $v$ are (non-zero) vectors. Note that matrix $u v^T$ is rank-$1$. Is there a formula for the matrix exponential $...
  • 11
1 vote
1 answer
86 views

Inverse of $UAA^TU^T$

Let $U\in \mathbb{R}^{n\times m}$, $n<m$ be a matrix with orthogonal rows, $UU^T=I$, and $A\in\mathbb{R}^{m\times k}$, $m<k$ be any general real matrix. What can I say on $(UAA^TU^T)^{-1}$ as a ...
0 votes
0 answers
19 views

maximization on trace of quadratic and linear terms under orthonormal constraints

How would one solve the following optimization problem: \begin{eqnarray} \max_{R: RR^{T}=I} Tr(M(RAR^{T}-KR^{T})) \end{eqnarray} where: M is a square, symmetric matrix with singular values 1 or 0, ...
  • 31
0 votes
2 answers
70 views

Gradient of the trace distance (Schatten $1$-norm) [closed]

Suppose that matrices $A$ and $B$ are Hermitian and positive semidefinite. How can I obtain the gradient of the trace distance between $A$ and $B$, i.e., $$C := \Vert A - B \Vert_1 := \mbox{Tr} \left( ...
2 votes
1 answer
40 views

How can apply coördinate descent method on the finding least norm solution of a linear system?

Suppose we have a linear system $Ax=b$, where $A\in\Bbb{R}^{m\times n}$ and $b\in\Bbb R^m$. Given the possibility of multiple solutions, I want to find a least norm solution for this system by solving ...
7 votes
2 answers
138 views

For matrix-valued functions, does $\frac d {dt} \exp(A(t)) = A'(t)\exp(A(t))$ imply $A'(t)A(t)=A(t)A'(t)$?

The converse (that $A$ and $A'$ commuting implies $(\exp(A)' = A'\exp(A)$) is easy to show from the series for $\exp$. In a class I'm TA for, one question on the students' exam was to find an example ...
0 votes
0 answers
13 views

Construct pseudospectrum for fourth-order matrix

For a normal fourth-order matrix A with eigenvalues 1, 2, 4, 8, construct pseudospectrum $\Lambda_{\varepsilon}(A)$ and structured pseudospectrum $\Lambda_{\varepsilon}(A;I,A)$, $\varepsilon = 1/4$.
0 votes
0 answers
39 views

The smallest positive eigenvalue of a special matrix

We know that $\kappa(A)=\lambda_{\mathrm{max}}(A^{\top}A)/\lambda_{\mathrm{min}}(A^{\top}A)$ is the condition number of $A$. Here, $m=m_1m_2$, $$ \mathbf{A}=\left[\begin{array}{llll} \mathbf{B} & &...
  • 85
1 vote
0 answers
31 views

Equivalence of two-norm between a matrix and its absolute form

Let $A$ a $n\times n$ real matrix, and denote by $|A|$ the matrix formed with the absolute values of the entries of A. I have proved that $||\; |A|\;||_2\le \sqrt{n}||A||_2$, using the identity $||A||...
1 vote
1 answer
39 views

Are there non-diagonal matrices in $\operatorname{SL}_2$ closest to $\sigma \operatorname{Id}$?

$\newcommand{\dist}{\operatorname{dist}}$ $\newcommand{\id}{\operatorname{Id}}$ $\newcommand{\diag}{\operatorname{diag}}$ $\newcommand{\SL}{\operatorname{SL}}$ $\newcommand{\SO}{\operatorname{SO}_2}$ $...
  • 24.3k
0 votes
2 answers
52 views

What's the derivative of an inner product with respect to the inner product matrix?

When you have an expression like this: $$g \left ( \mathbf{X} \right ) = \mathbf{a}^T \mathbf{X} \mathbf{b},$$ where $\mathbf{a} \in \mathbb{R}^d$, $\mathbf{b} \in \mathbb{R}^e$ and $\mathbf{X} \in \...
  • 101
-1 votes
0 answers
29 views

equilibria for a non-autonomous system, question and answer

I've stumbled upon a problem studying for my exams. I can't wrap my head around how they came about the answer [4.2;4.2]. The answer is given in the picture below. Could anyone help explain how you ...
-3 votes
0 answers
79 views

Determine the mapping matrices

$φ:R^4 → R^3, v\mapsto$ $$\begin{pmatrix} 1&3&2&0\\ 2&0&0&-3\\ -1&-1&4&5\end{pmatrix}\cdot v$$ We also have two basis $B$ for $R^4$ and $C$ for $R^3$ $B={(1\;-1\;0\...
user avatar
0 votes
0 answers
42 views

In a mathematical formula with “ max ” sign, how can I take the derivative and make the derivative zero to obtain the local minimum of the variable?

$$\begin{equation}\begin{split}\mathbf{W}^{(k+1)}&=\text{argmin}_{\mathbf{W}}\left\|\mathbf{F}-\mathbf{A\mathbf{W}}^{(k)}\right\|_F^2+\frac{\alpha_1}{2}\left\|\mathbf{B}\mathbf{W}^{(k)}-\mathbf{W}...
2 votes
1 answer
86 views

Prove a Matrix Property

I'm stuck with the following problem. Let be $W\in \mathbb{R}^{n\times n}$ matrix, $n\geq 2$. $W$ is such that: $W_{ij}\geq 0$ for all $0\leq i,j\leq n$. $W_{ii}=0$ for all $0\leq i\leq n$. $W$ is ...
0 votes
1 answer
33 views

Taking the derivative of a scalar function of an augmented vector with respect to a matrix

I am trying to take the derivative of a scalar function with respect to a matrix. The scalar function includes an augmented vector. The derivative I am interested in is: $$\frac{df([a|(\vec{b})^T\...
  • 3
-1 votes
1 answer
59 views

$\min_{\small X} \left\| X - Y\right\|_{2}^{2} + \left\| DX \right\|_{2}^{2}$ [closed]

Given that $\quad X, Y \in R^{N \times M}$ $\quad D \in R^{N \times N}$ Finding it difficult to proceed differentiation on these matrix equation. The above form can be rewritten as follows Equivalent ...
  • 3
0 votes
0 answers
22 views

"Gap" between the $x_i$ when it comes to maximizing $x^TAx$ over simplex for blockwise matrix?

We consider the local maximizer $x^TAx$ over simplex. $A$ is a symmetric $N \times N$ matrix with 4 blocks. First block $L \times L$ with elements $0 <a_{ij}<0.5$ Second block $K \times K$ ...
1 vote
1 answer
49 views

Question on matrix over matrix derivation

Consider recursive relations $$\mathbf{H}_k=\sigma(\mathbf{Z}_k),\ \mathbf{Z}_k=\mathbf{A}\mathbf{H}_{k-1}\mathbf{W}_k$$ where $\mathbf{Z}_k\in\mathbb{R}^{m\times n_k}$, $\mathbf{A}\in\mathbb{R}^{m\...
  • 441
1 vote
3 answers
69 views

Is there a closed-form analytical solution to: Maximize $y^T (X \beta) $ s.t. $(X \beta)^T (X \beta) = y^T y$.

Maximize $y^T (X \beta) $ s.t. $(X \beta)^T (X \beta) = y^T y$. Here $y$ is a known vector with size $n$ and $X$ is a known $n$ by $m$ matrix. $\beta$ is the unknown vector with size $m$ we want to ...
  • 1,411
0 votes
0 answers
16 views

A good way to measure similarity of self-similarity matrices of different sizes

In the data analysis and optimization problem I am currently working on, I want to use following equality constraint (or as additional objective function): For real-valued matrices $B$ and $X$ where $...
1 vote
1 answer
104 views

Chain rule for matrices in index notation

Consider recursive relations $$ \textbf{H}^t=\sigma(\textbf{Z}^t) $$ $$ \textbf{Z}^t=\textbf{A}\textbf{H}^{t-1}\textbf{W}^t $$ where $\textbf{Z}^t\in\mathbb{R}^{m\times n_t}$, $\textbf{A}\in\mathbb{R}^...
  • 441
1 vote
0 answers
50 views

Commutativity up to transposition

Suppose I have two square matrices $A, B$ for which the following property holds: $$AB= (BA)^\top$$ Linear time-invariant systems have the property $AB=BA$, i.e., it is said that they commute. In my ...
  • 745
1 vote
1 answer
43 views

Matrix derivation involving element-wise function

Let $$ \mathbf{H}=\sigma(\mathbf{Z}) $$ where $\sigma$ is an element-wise non-linear function, $\mathbf{H}\in\mathbb{R}^{m\times n}$, and $\mathbf{Z}\in\mathbb{R}^{m\times n}$. What is the index ...
  • 441
1 vote
2 answers
88 views

Proving the a matrix is bounded for every power

I want to show that the following matrix satisfies $\|A^n(z)\| \leq C$ for every $z \in \mathbb{C}$ with $Re(z) \leq 0$, for $n= 0,1,2,...$, and some constant $C$. How do I approach this kind of ...
  • 527
0 votes
1 answer
28 views

Trace -logarithm - matrix

How do I calculate this, $\beta$ is a parameter, $H$ a matrix: $(1-\beta \partial_\beta)\ln(Tr(e^{-\beta H})) = \ln(Tr(e^{-\beta H})) - \beta \frac{\partial_\beta Tr(e^{-\beta H})}{Tr(e^{-\beta H})}=?$...

1
2 3 4 5
72