Questions tagged [matrix-calculus]
Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.
2,553
questions
0
votes
0answers
20 views
Name for a matrix with sum of elements along every row/column is equal?
Is there a specific name for a symmetric matrix where sum of elements along every individual row is equal to the sum of elements along every individual column?
For example,
\begin{equation*}
A = \...
0
votes
1answer
21 views
Matrix Calculus: Derivative of double centered Euclidean distance matrix
I want to do this matrix calculus: Given, a distance matrix of squared Euclidean distances $D(X)_{n \times n}$ of $n$ points $X \in \mathbb{R}^k$, and given $C_n$, a centering matrix as defined in ...
0
votes
0answers
24 views
Schur complement question.
Good evening. Before start, I would like to pray for the end of the COVID19 virus soon.
I have a question related to the calculation of one Schur complement matrix.
Here is the matrix I'm interested ...
0
votes
1answer
49 views
Stuck on matrix derivative
I am stuck with this (probably simple) derivatives:
$$
\frac{\partial}{\partial X}Tr((A\odot(B^{T}XB))C)\;\;and \;\;\frac{\partial}{\partial X}Tr((A\odot(B^{T}XX^{T}B))C)
$$
where $A,B,C$ are ...
0
votes
1answer
27 views
Algorithms for computing $(I + \frac{Q}{n})^n$
I need to compute the matrix $(I + \frac{Q}{n})^n$ where $n = 2^k$, $k$ is integer. I want to know if there is any practical and effective algorithms.
1
vote
1answer
27 views
Complex derivative of Hadamard product inside Frobenius norm
I'm trying to find the complex derivative of
$$||R - P \circ \gamma \gamma ^H||_F ^2$$.
with respect to $\gamma$. I saw the post regarding the real counterpart of the same question here. However, ...
-1
votes
0answers
42 views
Derivative of a matrix with respect to another matrix [closed]
$A(t)$ is a real $n\times n$ matrix function on $t\in[0, T]$, let $f(A(t))=R^T(t)A(t)R(t)$, where $R(t)$ is an orthogonal matrix and also a function of $A(t)$. What is $\text{det}\left(\frac{\partial ...
0
votes
1answer
15 views
Integral of product of matrix exponentials
Is there an analytic expression for
$$
\int_{0}^{\infty} e^{-At}e^{-Bt} dt
$$
where $A$ and $B$ are non-commuting matrices?
0
votes
1answer
28 views
What are the meanings of matrix constructs of pre-multiplication of the matrix transpose and post-multiplication by the matrix itself.
In many books and articles I sometimes see similar matrix constructs, multiplication of the matrix transpose (or inverse), some other matrix, and then the first matrix itself, like this:
$M^T AM$ or $...
0
votes
1answer
31 views
Matrix non-singular proof
I have one question of how to derive the nonsingularity of one matrix. Here's the matrix I'm interested in:
\begin{align}
A = I + SHFG,
\end{align}
where $A \in \mathcal{R}^{m \times m}$, $I\in \...
4
votes
1answer
60 views
What is intuition behind matrix differentiation?
I understand the intuition behind normal function differentiation, it tells us how function varies when we wiggle the variable. But I don't understand what matrix differentiation tells us.
0
votes
0answers
24 views
Transformation Matrix Operations
Good morning,
I'm doing a matrix exercise for college. I need to get a transformation matrix that allows me to multiply the vertices of a rectangle and get the correct position of them from the ...
1
vote
0answers
26 views
Matrix calculus proof. Could someone help me please?
For the following proposition could someone perhaps explain to me moving from equation 45 to 46? I would be most grateful if you could do so with an example with simple matrices A and x.
Thanks ...
0
votes
1answer
39 views
Perturbing a matrix by $\epsilon$. Does the largest eigenvalue also changes a litte ? Or, it can change hugely [closed]
Suppose I perturb a matrix by $\epsilon$. Does the largest eigenvalue also changes a little ? Or, it can change hugely . Thanks.
2
votes
1answer
64 views
Calculate the gradient of a function over a matrix with element-wise terms
Consider the following problem
$$
J(v) = \frac{\lambda}{2}|| g - v ||_2^2 + \sum\limits_{i=1}^m\sum\limits_{j=1}^n \phi_\alpha((\delta_x^hv)_{i,j})+\phi_\alpha((\delta_y^hv)_{i,j})
$$
where $ g,v $ ...
2
votes
1answer
34 views
Minimizing $f(X) + \langle Y,X+X^T \rangle +\|X+X^T\|^2$
The optimization problem is as follows:
\begin{equation}
\min_{X \in \mathbb{R}^{n \times n}} ~f(X) + \langle Y, X+X^T \rangle + \|X+X^T\|^2,
\end{equation}
where $f \colon \mathbb{R}^{n \times n} \to ...
0
votes
1answer
15 views
Rearranging linear system containing a symmetric matrix
I'm trying to rearrange the following equation to get only Q on the LHS:
$$
A = -Q + t(Q M + M^T Q - kQ)
$$
Q is a symmetric matrix. A is a known symmetric matrix, M is an unknown non-symmetric ...
1
vote
0answers
49 views
Unique Solution for Square-root function on Matrices
Prove that for every n by n matrix M sufficiently close to the identity matrix there exists a square-root matrix (solution of $A^2 = M$) and the solution is unique if $A$ is required to be ...
0
votes
1answer
35 views
Derivative of $\text{Tr}[B X^T A X^{-1}]$
Let $A, B, X \in \mathbb{R}^{n \times n}$ and assume that $X^{-1}$ exists. Derive $\frac{\partial K}{\partial X}$ where
$K(X)= \text{Tr}[B X^T A X^{-1}]$
I have tried the following so far ($U = B X^...
0
votes
2answers
38 views
Derivative of Product of Matrix with Moore-Penrose Inverse
Assume that $A(x) \in M_{m,n}$ is given and depends on some $x \in \mathbb{R}$. Let $A^\dagger(x) = [A(x)^\star A(x)]^{-1} A^\star(x)$ be the Moore-Penrose pseudoinverse of $A(x)$.
Define $G(x) = A(x)...
0
votes
0answers
14 views
A system of SDEs
The question:
Suppose I have a $d$-dimensional stochastic process $X(t)$ such that
$$ dX = (\alpha + AX)dt + \sigma dW(t) $$
where $A$ and $\sigma$ are constant $d$-by-$d$ matrices, $\alpha$ is a $d$...
0
votes
0answers
13 views
From derivatives to gradients in backpropagation
Consider the following explanation of backpropagation from Wikipedia:
Given an inputāoutput pair {\displaystyle (x,y)}(x,y), the loss is:
$ C(y,f^{L}(W^{L}f^{L-1}(W^{L-1}\cdots f^{1}(W^{1}x)\...
1
vote
0answers
20 views
Analytic eigenvalues
Is the following conclusion true?
Suppose $A,B$ are $n\times n$ complex Hermitian matrices. Then there exists real analytic functions $\lambda_i:\mathbb R\to \mathbb R$ where $1\leq i\leq n $ such ...
0
votes
2answers
65 views
Taylor Expansion of Logarithm of Determinant near Identity for Non-Diagonalizable Matrix
I have been working on a problem where I need to Taylor expand an expression of the form $\log \det(I-A)$ in terms of traces of the matrices $A^m$ for $m \in \mathbb N$, where $A$ is a general $n \...
2
votes
2answers
199 views
Taylor expansion of a function of a symmetric matrix
First of all let me tell you that the answer to this question is likely to confirm a not-so-minor error in a very popular (and excellent) textbook on optimization, as you'll see below.
Background
...
2
votes
0answers
20 views
Diagonalizable matrix over R
I had study about solving linear difference equation system by using results of linear algebra.
But I have a problem with the following exercise:
Suppose that $A$ is a matrix with real entries, $A$ ...
0
votes
1answer
28 views
Logarithm of a matrix with zero diagonal elements.
I want to calculate the natural logarithm of this matrix$$A=\begin{bmatrix}0&0&0&0\\0&\frac{1}{2}&\frac{i}{2}&0\\0&-\frac{i}{2}&\frac{1}{2}&0\\0&0&0&0\...
0
votes
1answer
53 views
What are some strategies for computing derivatives in matrix calculus?
In single variable calculus, the chain, product, and power rules are very straightforward. In vector calculus, it's a bit more involved.
I frequently find myself having to do a lot of work to arrive ...
0
votes
0answers
35 views
Solving a matrix equation given an eigenvector
If $v(z=0)$ is chosen to be an eigenvector of $A$ describe the solution evaluated for $v(nĪ“)$, for any integer $n$ for the following:
$$(NI-A)v(z+Ī“)=(NI+A)v(z)$$
where $N$ is an arbitrary number. ...
0
votes
2answers
57 views
Differential of a matrix function: $f:\mathbb{R}^{n \times m} \rightarrow \mathbb{R}^{m \times 1}$ $f(A) = A^T\cdot \vec{v}$
I wish to calculate the differential of a function: $f(A) = X^T\cdot \vec{v}$ when $A\in \mathbb{R}^{n \times m}$ with respect to $A$.
Since this is a linear function, if we think about $D\in \...
0
votes
0answers
29 views
Can someone help me in the solution … [duplicate]
So this question was asked in an online quiz.
The definiteness of the quadratic form
Q(x) = 8$x_1^2$ + 7$x_2^2$ + 3$x_3^2$ - 12$x_1x_2$ - 8$x_2x_3$ + 4$x_3x_2$
I made the following matrix:
\begin{...
3
votes
0answers
33 views
Derivative of $\log^2(\det(A))$ w.r.t. matrix A
Given that $A\in\mathbb{R}^{n\times n}$, is there an explicit matrix expression for
$$\frac{d}{dA}\log^2(\det(A))$$
(in numerator layout format)?
Attempt
$$=2\log(\det(A))\frac{d}{dA}\log(\det(A))$...
2
votes
3answers
63 views
Derivative of squared norm of component of a matrix perpendicular to identity matrix, with respect to the original matrix
Let $J\in\mathbb{R}^{n\times n}$
What is the derivative (with respect to $J$) of the squared norm of the component of $J$ that is orthogonal to $I$ (the identity matrix)?
Attempt
$J$'s projection ...
0
votes
1answer
22 views
trace and derivative: understanding $\text{tr}\left(e_j e_i^T B^T B \right) = \langle B e_i e_j^T, B \rangle $
where $B$ is $n \times n$ matrix
Can someone help me understand the above equality. what kind of inner product we are using here?
Also, why do we have $\frac{d}{dB}\left(\langle B e_i e_j^T, B \...
2
votes
1answer
32 views
Derivative of L1 norm of Hadamard product
I am trying to find the derivative of $f(B)=\lambda\Vert W \bigodot B \Vert_1 + \frac{\rho}{2}\Vert A-B \Vert_F^2 + tr(\Delta^T(A-B))$ with respect to B.
where B is (nĆnļ¼matrix, W is (nĆnļ¼constant ...
0
votes
1answer
70 views
Least Squares: Derivation of Normal Equations with Chain Rule (Revisited)
My question pertains to someone else's answered question that has made me curious. The OP wanted to differentiate the following using the chain rule:
$$
J(\theta)=\frac12(X\theta-y)^T(X\theta - y)
$$
...
0
votes
2answers
59 views
What does $[T]_{\cal BB}$ mean?
I have been given a matrix P where the columns represent a basis in B. I have also been given a matrix A which is the standard matrix for T. I am then supposed to calculate the matrix for T relative ...
1
vote
1answer
58 views
How to write the first term of Taylor expansion for a function matrix?
Assume $F(A)=\left\| A^N v-w \right\|^2$ where $A$ is a square matrix and $v$ and $w$ are two constant vectors and $N$ is a positive integer. Also, the norm is the Euclidean norm. How to write the ...
1
vote
1answer
58 views
Minimizing $\| x A - B\|_F^2$ With a Constraint
I have previously asked an optimization question Here. I will reiterate the question and simply add a constraint to it:
I have 2 known grayscale images (256Ć256 matrices) $A$ and $B$ and want to find ...
1
vote
1answer
45 views
Vector calculus notation
Say we have a $n \times m$ matrix $\boldsymbol{X}$. When we see $\boldsymbol{X}^T_i$, is this typically referring to
(1) the i-th row of $\boldsymbol{X}^T$, which is a column vector. So $\boldsymbol{...
2
votes
0answers
53 views
Show that $p^T \Sigma^{-1} p \leq k^2$ is equivalent to $w^Tpp^Tw \leq k^2 w^T \Sigma w$
I am trying to understand a research paper 1 (Eq 11 and Eq 10) which states the solution to the following 2 optimization problems are equivalent.
$$\begin{array}{ll} & \min_{p} p^Tw \\ & \...
0
votes
0answers
32 views
How to derive the Lyapunov Equation?
I have a linear dynamical system:
$\dfrac{dx}{dt} = Ax$ with $A \in \mathbb{R}^{n\times n}$ and $x \in \mathbb{R}^{n\times1}$.
I consider a quadratic function $V(x) = x^T P x$ where $P \in \mathbb{R}...
0
votes
1answer
39 views
Derivative of $f(P^{-1}HP)$ w.r.t. H
Given
$f:\mathbb{R}^{n\times n}\rightarrow\mathbb{R}$
$H\in\mathbb{R}^{n\times n}$ is a variable
$P\in\mathbb{R}^{n\times n}$ is a constant
$f':\mathbb{R}^{n\times n}\rightarrow\mathbb{R}^{n\times n}$...
0
votes
0answers
18 views
Maximize quadratic form under the constraint of a unit hypersphere
I'm trying to maximize $f(\vec{v}) =\vec{v}^TA\vec{v}$ ($A$ is a symmetric matrix, e.g., covariance matrix) under the constraint that $||\vec{v}||^2=\vec{v}^T\vec{v}=1$. I haven't taken any matrix ...
1
vote
1answer
42 views
Derivative of $P^{-1}HP$ w.r.t. $H$
Is there a matrix expression for $\frac{d}{dH}P^{-1}HP$ (for constant square matrix $P$)?
Background (if necessary)
I seek to design an objective function (and its gradient) over the 8D space of ...
1
vote
1answer
34 views
Taylor-formular for Matrices
Let $A\in\mathbb{C}^{n\times n}$. What does it mean to make a Taylor expansion of $A$ around a point? I assume that the induced linear map is what should be expanded, but I have never seen a Taylor ...
0
votes
0answers
20 views
2 Coin tossed using Markov Chains
A sequence of experiments is executed, in each of the two coins they are thrown. Let $S_1$ indicate that two suns came out, $S_2$ indicate that a sun and an eagle came out, and $S_3$ indicate that two ...
0
votes
1answer
40 views
Interpret a matrix as a function
I have a simple matrix that I don't understand how to write as a function.
I have attached a picture of the matrix and have come up with two different answers, A or B.
Can anyone give me a hint on ...
1
vote
0answers
40 views
Matrix inverse with diagonal Perturbations
Consider
$$ P:= (X^TX+D_1^{-1})^{-1} - (X^TX+D_2^{-1})^{-1} $$
$$ Q:= (X^TX+D_1^{-1})^{-1/2} - (X^TX+D_2^{-1})^{-1/2} $$
$X$ is an $n \times p$ matrix with $p > n$ and $D_1, D_2$ are diagonal ...
0
votes
2answers
42 views
Square matrix 0 x 0?
Can you consider a 0x0 matrix as a square matrix, I can't find the precise definition. I need it for my programming assignment for throwing exceptions.
