Questions tagged [matrix-calculus]
Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.
3,591
questions
1
vote
0
answers
34
views
Determine the Hessian of this function?
Given the function below, I would like to determine the Hessian with respect to $\mathbf{x}$, which will result in a $2 \times 2$ matrix. Note: $n$ and $\mathbf{z}$ are constants with respect to $\...
- 452
2
votes
1
answer
22
views
Continuity of maximum eigenvalue function
Let $f:S^n_{+}(\mathbb{R})\rightarrow \mathbb{R}_+$ be defined as $$f(A)=\lambda_{\max}(A),$$ where $S^n_{+}(\mathbb{R})$ is set of all positive definite $n\times n$ matrices.
What can we say about ...
- 175
0
votes
1
answer
28
views
Derivative of a vector x wrt itself
I am attempting to better understand matrix calculus. Given a vector $x$, I understand that:
$\frac{\mathrm{d} x^t}{\mathrm{d} x}= I$ but struggle to think about $\frac{\mathrm{d} x}{\mathrm{d} x}$. ...
- 27
-1
votes
0
answers
42
views
How to compute gradient of a function of matrix? [closed]
Write the explicit formula of the gradient of $$ E(u) = \sum_{i=2}^{n-1}\sum_{j=2}^{m-1} \sin\left(\left(u[i+1, j] - u[i,j-1]\right)^2 + \epsilon\right) $$ with respect to the variable $u$, which is ...
- 1
0
votes
0
answers
8
views
Vector by tt-Matrix multiplication [closed]
Let's say:
Y is a matrix with size of [1,M]
X is a matrix with size of [1,N]
Z is a matrix with size of [N,M]
It is needed to perform the product of X and Z in order to generate Y values (e.g., Y = ...
- 1
1
vote
1
answer
25
views
Stuck in matrix manipulation and rearrangement
We have that
$A$ is a symmetric matrix $n\times n$, $b$ is a $n\times 1$ vector, $C$ is a scalar, and $d$ is a $n\times 1$ vector.
The expression
$\left(Ad\right)'A^{-1}\left(Ad\right)-2b'A^{-1}Ad+C$
...
- 47
-4
votes
0
answers
22
views
How do I take derivative of a Matrix [closed]
I have a Matrix A and every element in A is a function of t or zero.
A = [[2*t^2, 0],
[0, t+5]]
for this 22 matrix, how can I take the derivative of it?
0
votes
0
answers
29
views
How to solve AX=B for A when A, X and B are (2x2) matrices?
I've just been trying to solve this problem but I'm struggling, I've looked online and all the examples solve for X which seems more simple.
What I'm trying to do is calculate a calibration matrix for ...
- 1
0
votes
1
answer
17
views
Exponential matrix derivative to find the Hessian matrix of negative binomial regression
I am looking for the hessian matrix of the log likelihood function of negative binomial regression
$$l\left( \cdot \right) =\sum ^{n}_{i=1}y_{i}\ln \left( \dfrac{\alpha \exp \left( x_{i}^{T}\beta \...
- 1
0
votes
1
answer
37
views
Gradient of $ \left( \mathbf{w}^T \mathbf{x} \right)^2 - 2\mathbf{w}^T \mathbf{x}$ w.r.t $\mathbf{w}$
That's it. So far I've tried the following but I'm not certain if I'm allowed to do that.
\begin{align}
\frac{d\left( \left( \mathbf{w}^T \mathbf{x} \right)^2 - 2\mathbf{w}^T \mathbf{x} \right)}{d\...
- 45
3
votes
0
answers
94
views
Show that $\det(xA+yB+zI_{n})=\det(yA+xB+zI_{n})$
We have $A$ and $B$ $(n×n)$ matrices with complex entries. We know that $A-B=AB-BA$. Show that $$\det(xA+yB+zI_{n})=\det(yA+xB+zI_{n})$$
for every $x,y,z$ complex numbers with $x+y≠0$.
We can see that ...
- 773
2
votes
0
answers
41
views
What's the geometric meaning of total positivity?
Is there a geometric meaning of total positivity? I wanted to thought of it as a measure of volume but there wasn't a derivative definition associated with the total positivity. But does it define ...
1
vote
0
answers
37
views
Response function in weakly coupled differential equations
I have the following system of differential equations:
\begin{equation}
\frac{d}{dt} \, \begin{pmatrix} x_1 (t) \\ x_2 (t) \\ x_3(t) \\ x_4(t) \\ \vdots \\ x_M (t) \end{pmatrix} = \begin{pmatrix} f_1(...
- 386
0
votes
0
answers
32
views
The differential of trace function
This answer states that the differential of a trace function is given by
$d\text{Tr}(f(X)) = f'(X^T):dX$ where the colon $A:B$ means $\text{Tr}(A^TB)$.
How do we prove it? Does any know any reference/...
- 103
0
votes
0
answers
37
views
Matrix derivatives for ML
Assume that our data is distributed according to a $\underline d$ dimensional multivariate Gaussian with $\bar \mu$ mean and $\Sigma$ covariance matrix: $$(\mathbf x_1, \dots, \mathbf x_n) \sim \...
0
votes
0
answers
53
views
Remove a vector from a subspace
Suppose I have a $N \times N$ matrix $A$ which is a projection matrix on the column space of another matrix $B=[b_1, \ldots, b_n], n<N $, that is, $A=BB^\dagger$, with $^\dagger$ denoting the ...
- 745
7
votes
2
answers
210
views
Taylor series of a matrix exponential
I am looking to minimize the value of:
$$g(t)=\mathrm{Tr}\left[\exp(X+tY)\right]$$
where both $X$ and $Y$ are symmetrical matrices with real coefficients. In general, $X$ and $Y$ do not commute so $\...
- 1,835
0
votes
0
answers
15
views
PCA proof, trace move
In the PCA proof, I see this move:
What is the "trace" algebra/rule that allow this trace-move?
- 103
1
vote
1
answer
98
views
Deriving backpropagation equations - vectorization (regression)
I have a huge problem trying to derive the backpropagation equations. All the solutions I've found online are not detailed as I'd like, hence I'm here asking your help.
First of all sorry for this ...
- 111
0
votes
0
answers
47
views
+50
Functional derivative of $u \mapsto R\left(\int_{\Theta} \phi(\theta) \; \text{d}u(\theta)\right) + \lambda u(\Theta)$ for matrix-valued measure $u$
Let $\Theta$ be a closed connected manifold, $H$ and $F$ Hilbert spaces and $R \colon H \to \mathbb R$ and $\phi \colon \Theta \to F$ smooth.
What is the functional derivative of
$$f(u) := R\left(\...
- 4,339
0
votes
1
answer
43
views
Definition of Derivative of Matrix
Let's assume A is $n\times 1$ constants, $X$ is $n\times 1$ vector. Does derivative of transpose(A)* X on X should be transpose(A) instead of A?
I saw both transpose(A) and A from different resources ...
- 31
1
vote
2
answers
26
views
Is the derivative of $\pmb x^\top (I+GB)^{-1} (I+GB)^{-\top}\pmb x$ with respect to $B$ a 4th-order tensor?
Is the derivative of $\pmb x^\top (I+GB)^{-\top} (I+GB)^{-1}\pmb x$ with respect to $B$ a 4th-order tensor?
Where $\pmb x$ is a vector, and $B$ is a matrix.
I followed the procedures in What is the ...
5
votes
2
answers
132
views
Matrix derivative of $f(X^T Y)$ w.r.t X
Given $X\in \mathbb{R}^{m\times n}$, $Y\in \mathbb{R}^{m\times k}$, $X^\top Y=Z\in \mathbb{R}^{n\times k}$,
and $f:\mathbb{R}^{n\times k} \to \mathbb{R}$, we have the following:
\begin{equation}
f(X^\...
- 165
0
votes
2
answers
36
views
Hessian Matrix of Matrix Product
I am not sure how I can compute the Hessian Matrix of a trace of the matrix such as this:
$$ f(w) := \operatorname{tr} \left( B w w^T A \right) $$
where $A$ and $B$ are $n \times n$ square matrices ...
- 1
0
votes
1
answer
35
views
Matrix exponential of the sum of a diagonal matrix and a rank-$1$ matrix
Suppose we have the matrix $D + u v^T$, where $D$ is a diagonal matrix and both $u$ and $v$ are (non-zero) vectors. Note that matrix $u v^T$ is rank-$1$. Is there a formula for the matrix exponential $...
- 11
1
vote
1
answer
86
views
Inverse of $UAA^TU^T$
Let $U\in \mathbb{R}^{n\times m}$, $n<m$ be a matrix with orthogonal rows, $UU^T=I$, and $A\in\mathbb{R}^{m\times k}$, $m<k$ be any general real matrix.
What can I say on $(UAA^TU^T)^{-1}$ as a ...
- 351
0
votes
0
answers
19
views
maximization on trace of quadratic and linear terms under orthonormal constraints
How would one solve the following optimization problem:
\begin{eqnarray}
\max_{R: RR^{T}=I} Tr(M(RAR^{T}-KR^{T}))
\end{eqnarray}
where:
M is a square, symmetric matrix with singular values 1 or 0, ...
- 31
0
votes
2
answers
70
views
Gradient of the trace distance (Schatten $1$-norm) [closed]
Suppose that matrices $A$ and $B$ are Hermitian and positive semidefinite. How can I obtain the gradient of the trace distance between $A$ and $B$, i.e., $$C := \Vert A - B \Vert_1 := \mbox{Tr} \left( ...
- 115
2
votes
1
answer
40
views
How can apply coördinate descent method on the finding least norm solution of a linear system?
Suppose we have a linear system $Ax=b$, where $A\in\Bbb{R}^{m\times n}$ and $b\in\Bbb R^m$. Given the possibility of multiple solutions, I want to find a least norm solution for this system by solving ...
- 55
7
votes
2
answers
138
views
For matrix-valued functions, does $\frac d {dt} \exp(A(t)) = A'(t)\exp(A(t))$ imply $A'(t)A(t)=A(t)A'(t)$?
The converse (that $A$ and $A'$ commuting implies $(\exp(A)' = A'\exp(A)$) is easy to show from the series for $\exp$. In a class I'm TA for, one question on the students' exam was to find an example ...
- 5,482
0
votes
0
answers
13
views
Construct pseudospectrum for fourth-order matrix
For a normal fourth-order matrix A with eigenvalues 1, 2, 4, 8, construct pseudospectrum $\Lambda_{\varepsilon}(A)$ and structured pseudospectrum $\Lambda_{\varepsilon}(A;I,A)$, $\varepsilon = 1/4$.
0
votes
0
answers
39
views
The smallest positive eigenvalue of a special matrix
We know that $\kappa(A)=\lambda_{\mathrm{max}}(A^{\top}A)/\lambda_{\mathrm{min}}(A^{\top}A)$ is the condition number of $A$.
Here,
$m=m_1m_2$,
$$
\mathbf{A}=\left[\begin{array}{llll}
\mathbf{B} & &...
- 85
1
vote
0
answers
31
views
Equivalence of two-norm between a matrix and its absolute form
Let $A$ a $n\times n$ real matrix, and denote by $|A|$ the matrix formed with the absolute values of the entries of A. I have proved that $||\; |A|\;||_2\le \sqrt{n}||A||_2$, using the identity $||A||...
- 11
1
vote
1
answer
39
views
Are there non-diagonal matrices in $\operatorname{SL}_2$ closest to $\sigma \operatorname{Id}$?
$\newcommand{\dist}{\operatorname{dist}}$
$\newcommand{\id}{\operatorname{Id}}$
$\newcommand{\diag}{\operatorname{diag}}$
$\newcommand{\SL}{\operatorname{SL}}$
$\newcommand{\SO}{\operatorname{SO}_2}$
$...
- 24.3k
0
votes
2
answers
52
views
What's the derivative of an inner product with respect to the inner product matrix?
When you have an expression like this:
$$g \left ( \mathbf{X} \right ) = \mathbf{a}^T \mathbf{X} \mathbf{b},$$
where $\mathbf{a} \in \mathbb{R}^d$, $\mathbf{b} \in \mathbb{R}^e$ and $\mathbf{X} \in \...
- 101
-1
votes
0
answers
29
views
equilibria for a non-autonomous system, question and answer
I've stumbled upon a problem studying for my exams. I can't wrap my head around how they came about the answer [4.2;4.2].
The answer is given in the picture below.
Could anyone help explain how you ...
- 41
-3
votes
0
answers
79
views
Determine the mapping matrices
$φ:R^4 → R^3,
v\mapsto$
$$\begin{pmatrix} 1&3&2&0\\ 2&0&0&-3\\ -1&-1&4&5\end{pmatrix}\cdot v$$
We also have two basis $B$ for $R^4$ and $C$ for $R^3$
$B={(1\;-1\;0\...
user1127565
0
votes
0
answers
42
views
In a mathematical formula with “ max ” sign, how can I take the derivative and make the derivative zero to obtain the local minimum of the variable?
$$\begin{equation}\begin{split}\mathbf{W}^{(k+1)}&=\text{argmin}_{\mathbf{W}}\left\|\mathbf{F}-\mathbf{A\mathbf{W}}^{(k)}\right\|_F^2+\frac{\alpha_1}{2}\left\|\mathbf{B}\mathbf{W}^{(k)}-\mathbf{W}...
2
votes
1
answer
86
views
Prove a Matrix Property
I'm stuck with the following problem.
Let be $W\in \mathbb{R}^{n\times n}$ matrix, $n\geq 2$. $W$ is such that:
$W_{ij}\geq 0$ for all $0\leq i,j\leq n$.
$W_{ii}=0$ for all $0\leq i\leq n$.
$W$ is ...
- 109
0
votes
1
answer
33
views
Taking the derivative of a scalar function of an augmented vector with respect to a matrix
I am trying to take the derivative of a scalar function with respect to a matrix. The scalar function includes an augmented vector. The derivative I am interested in is:
$$\frac{df([a|(\vec{b})^T\...
- 3
-1
votes
1
answer
59
views
$\min_{\small X} \left\| X - Y\right\|_{2}^{2} + \left\| DX \right\|_{2}^{2}$ [closed]
Given that
$\quad X, Y \in R^{N \times M}$
$\quad D \in R^{N \times N}$
Finding it difficult to proceed differentiation on these matrix equation.
The above form can be rewritten as follows
Equivalent ...
- 3
0
votes
0
answers
22
views
"Gap" between the $x_i$ when it comes to maximizing $x^TAx$ over simplex for blockwise matrix?
We consider the local maximizer $x^TAx$ over simplex.
$A$ is a symmetric $N \times N$ matrix with 4 blocks.
First block $L \times L$ with elements $0 <a_{ij}<0.5$
Second block $K \times K$ ...
- 21
1
vote
1
answer
49
views
Question on matrix over matrix derivation
Consider recursive relations
$$\mathbf{H}_k=\sigma(\mathbf{Z}_k),\ \mathbf{Z}_k=\mathbf{A}\mathbf{H}_{k-1}\mathbf{W}_k$$
where $\mathbf{Z}_k\in\mathbb{R}^{m\times n_k}$, $\mathbf{A}\in\mathbb{R}^{m\...
- 441
1
vote
3
answers
69
views
Is there a closed-form analytical solution to: Maximize $y^T (X \beta) $ s.t. $(X \beta)^T (X \beta) = y^T y$.
Maximize $y^T (X \beta) $ s.t. $(X \beta)^T (X \beta) = y^T y$.
Here $y$ is a known vector with size $n$ and $X$ is a known $n$ by $m$ matrix. $\beta$ is the unknown vector with size $m$ we want to ...
- 1,411
0
votes
0
answers
16
views
A good way to measure similarity of self-similarity matrices of different sizes
In the data analysis and optimization problem I am currently working on, I want to use following equality constraint (or as additional objective function):
For real-valued matrices $B$ and $X$ where $...
- 701
1
vote
1
answer
104
views
Chain rule for matrices in index notation
Consider recursive relations
$$
\textbf{H}^t=\sigma(\textbf{Z}^t)
$$
$$
\textbf{Z}^t=\textbf{A}\textbf{H}^{t-1}\textbf{W}^t
$$
where $\textbf{Z}^t\in\mathbb{R}^{m\times n_t}$, $\textbf{A}\in\mathbb{R}^...
- 441
1
vote
0
answers
50
views
Commutativity up to transposition
Suppose I have two square matrices $A, B$ for which the following property holds:
$$AB= (BA)^\top$$
Linear time-invariant systems have the property $AB=BA$, i.e., it is said that they commute. In my ...
- 745
1
vote
1
answer
43
views
Matrix derivation involving element-wise function
Let
$$
\mathbf{H}=\sigma(\mathbf{Z})
$$
where $\sigma$ is an element-wise non-linear function, $\mathbf{H}\in\mathbb{R}^{m\times n}$, and $\mathbf{Z}\in\mathbb{R}^{m\times n}$. What is the index ...
- 441
1
vote
2
answers
88
views
Proving the a matrix is bounded for every power
I want to show that the following matrix satisfies $\|A^n(z)\| \leq C$ for every $z \in \mathbb{C}$ with $Re(z) \leq 0$, for $n= 0,1,2,...$, and some constant $C$. How do I approach this kind of ...
- 527
0
votes
1
answer
28
views
Trace -logarithm - matrix
How do I calculate this, $\beta$ is a parameter, $H$ a matrix:
$(1-\beta \partial_\beta)\ln(Tr(e^{-\beta H})) = \ln(Tr(e^{-\beta H})) - \beta \frac{\partial_\beta Tr(e^{-\beta H})}{Tr(e^{-\beta H})}=?$...
- 25