Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Filter by
Sorted by
Tagged with
0
votes
0answers
20 views

Name for a matrix with sum of elements along every row/column is equal?

Is there a specific name for a symmetric matrix where sum of elements along every individual row is equal to the sum of elements along every individual column? For example, \begin{equation*} A = \...
0
votes
1answer
21 views

Matrix Calculus: Derivative of double centered Euclidean distance matrix

I want to do this matrix calculus: Given, a distance matrix of squared Euclidean distances $D(X)_{n \times n}$ of $n$ points $X \in \mathbb{R}^k$, and given $C_n$, a centering matrix as defined in ...
0
votes
0answers
24 views

Schur complement question.

Good evening. Before start, I would like to pray for the end of the COVID19 virus soon. I have a question related to the calculation of one Schur complement matrix. Here is the matrix I'm interested ...
0
votes
1answer
49 views

Stuck on matrix derivative

I am stuck with this (probably simple) derivatives: $$ \frac{\partial}{\partial X}Tr((A\odot(B^{T}XB))C)\;\;and \;\;\frac{\partial}{\partial X}Tr((A\odot(B^{T}XX^{T}B))C) $$ where $A,B,C$ are ...
0
votes
1answer
27 views

Algorithms for computing $(I + \frac{Q}{n})^n$

I need to compute the matrix $(I + \frac{Q}{n})^n$ where $n = 2^k$, $k$ is integer. I want to know if there is any practical and effective algorithms.
1
vote
1answer
27 views

Complex derivative of Hadamard product inside Frobenius norm

I'm trying to find the complex derivative of $$||R - P \circ \gamma \gamma ^H||_F ^2$$. with respect to $\gamma$. I saw the post regarding the real counterpart of the same question here. However, ...
-1
votes
0answers
42 views

Derivative of a matrix with respect to another matrix [closed]

$A(t)$ is a real $n\times n$ matrix function on $t\in[0, T]$, let $f(A(t))=R^T(t)A(t)R(t)$, where $R(t)$ is an orthogonal matrix and also a function of $A(t)$. What is $\text{det}\left(\frac{\partial ...
0
votes
1answer
15 views

Integral of product of matrix exponentials

Is there an analytic expression for $$ \int_{0}^{\infty} e^{-At}e^{-Bt} dt $$ where $A$ and $B$ are non-commuting matrices?
0
votes
1answer
28 views

What are the meanings of matrix constructs of pre-multiplication of the matrix transpose and post-multiplication by the matrix itself.

In many books and articles I sometimes see similar matrix constructs, multiplication of the matrix transpose (or inverse), some other matrix, and then the first matrix itself, like this: $M^T AM$ or $...
0
votes
1answer
31 views

Matrix non-singular proof

I have one question of how to derive the nonsingularity of one matrix. Here's the matrix I'm interested in: \begin{align} A = I + SHFG, \end{align} where $A \in \mathcal{R}^{m \times m}$, $I\in \...
4
votes
1answer
60 views

What is intuition behind matrix differentiation?

I understand the intuition behind normal function differentiation, it tells us how function varies when we wiggle the variable. But I don't understand what matrix differentiation tells us.
0
votes
0answers
24 views

Transformation Matrix Operations

Good morning, I'm doing a matrix exercise for college. I need to get a transformation matrix that allows me to multiply the vertices of a rectangle and get the correct position of them from the ...
1
vote
0answers
26 views

Matrix calculus proof. Could someone help me please?

For the following proposition could someone perhaps explain to me moving from equation 45 to 46? I would be most grateful if you could do so with an example with simple matrices A and x. Thanks ...
0
votes
1answer
39 views

Perturbing a matrix by $\epsilon$. Does the largest eigenvalue also changes a litte ? Or, it can change hugely [closed]

Suppose I perturb a matrix by $\epsilon$. Does the largest eigenvalue also changes a little ? Or, it can change hugely . Thanks.
2
votes
1answer
64 views

Calculate the gradient of a function over a matrix with element-wise terms

Consider the following problem $$ J(v) = \frac{\lambda}{2}|| g - v ||_2^2 + \sum\limits_{i=1}^m\sum\limits_{j=1}^n \phi_\alpha((\delta_x^hv)_{i,j})+\phi_\alpha((\delta_y^hv)_{i,j}) $$ where $ g,v $ ...
2
votes
1answer
34 views

Minimizing $f(X) + \langle Y,X+X^T \rangle +\|X+X^T\|^2$

The optimization problem is as follows: \begin{equation} \min_{X \in \mathbb{R}^{n \times n}} ~f(X) + \langle Y, X+X^T \rangle + \|X+X^T\|^2, \end{equation} where $f \colon \mathbb{R}^{n \times n} \to ...
0
votes
1answer
15 views

Rearranging linear system containing a symmetric matrix

I'm trying to rearrange the following equation to get only Q on the LHS: $$ A = -Q + t(Q M + M^T Q - kQ) $$ Q is a symmetric matrix. A is a known symmetric matrix, M is an unknown non-symmetric ...
1
vote
0answers
49 views

Unique Solution for Square-root function on Matrices

Prove that for every n by n matrix M sufficiently close to the identity matrix there exists a square-root matrix (solution of $A^2 = M$) and the solution is unique if $A$ is required to be ...
0
votes
1answer
35 views

Derivative of $\text{Tr}[B X^T A X^{-1}]$

Let $A, B, X \in \mathbb{R}^{n \times n}$ and assume that $X^{-1}$ exists. Derive $\frac{\partial K}{\partial X}$ where $K(X)= \text{Tr}[B X^T A X^{-1}]$ I have tried the following so far ($U = B X^...
0
votes
2answers
38 views

Derivative of Product of Matrix with Moore-Penrose Inverse

Assume that $A(x) \in M_{m,n}$ is given and depends on some $x \in \mathbb{R}$. Let $A^\dagger(x) = [A(x)^\star A(x)]^{-1} A^\star(x)$ be the Moore-Penrose pseudoinverse of $A(x)$. Define $G(x) = A(x)...
0
votes
0answers
14 views

A system of SDEs

The question: Suppose I have a $d$-dimensional stochastic process $X(t)$ such that $$ dX = (\alpha + AX)dt + \sigma dW(t) $$ where $A$ and $\sigma$ are constant $d$-by-$d$ matrices, $\alpha$ is a $d$...
0
votes
0answers
13 views

From derivatives to gradients in backpropagation

Consider the following explanation of backpropagation from Wikipedia: Given an input–output pair {\displaystyle (x,y)}(x,y), the loss is: $ C(y,f^{L}(W^{L}f^{L-1}(W^{L-1}\cdots f^{1}(W^{1}x)\...
1
vote
0answers
20 views

Analytic eigenvalues

Is the following conclusion true? Suppose $A,B$ are $n\times n$ complex Hermitian matrices. Then there exists real analytic functions $\lambda_i:\mathbb R\to \mathbb R$ where $1\leq i\leq n $ such ...
0
votes
2answers
65 views

Taylor Expansion of Logarithm of Determinant near Identity for Non-Diagonalizable Matrix

I have been working on a problem where I need to Taylor expand an expression of the form $\log \det(I-A)$ in terms of traces of the matrices $A^m$ for $m \in \mathbb N$, where $A$ is a general $n \...
2
votes
2answers
199 views

Taylor expansion of a function of a symmetric matrix

First of all let me tell you that the answer to this question is likely to confirm a not-so-minor error in a very popular (and excellent) textbook on optimization, as you'll see below. Background ...
2
votes
0answers
20 views

Diagonalizable matrix over R

I had study about solving linear difference equation system by using results of linear algebra. But I have a problem with the following exercise: Suppose that $A$ is a matrix with real entries, $A$ ...
0
votes
1answer
28 views

Logarithm of a matrix with zero diagonal elements.

I want to calculate the natural logarithm of this matrix$$A=\begin{bmatrix}0&0&0&0\\0&\frac{1}{2}&\frac{i}{2}&0\\0&-\frac{i}{2}&\frac{1}{2}&0\\0&0&0&0\...
0
votes
1answer
53 views

What are some strategies for computing derivatives in matrix calculus?

In single variable calculus, the chain, product, and power rules are very straightforward. In vector calculus, it's a bit more involved. I frequently find myself having to do a lot of work to arrive ...
0
votes
0answers
35 views

Solving a matrix equation given an eigenvector

If $v(z=0)$ is chosen to be an eigenvector of $A$ describe the solution evaluated for $v(nδ)$, for any integer $n$ for the following: $$(NI-A)v(z+δ)=(NI+A)v(z)$$ where $N$ is an arbitrary number. ...
0
votes
2answers
57 views

Differential of a matrix function: $f:\mathbb{R}^{n \times m} \rightarrow \mathbb{R}^{m \times 1}$ $f(A) = A^T\cdot \vec{v}$

I wish to calculate the differential of a function: $f(A) = X^T\cdot \vec{v}$ when $A\in \mathbb{R}^{n \times m}$ with respect to $A$. Since this is a linear function, if we think about $D\in \...
0
votes
0answers
29 views

Can someone help me in the solution … [duplicate]

So this question was asked in an online quiz. The definiteness of the quadratic form Q(x) = 8$x_1^2$ + 7$x_2^2$ + 3$x_3^2$ - 12$x_1x_2$ - 8$x_2x_3$ + 4$x_3x_2$ I made the following matrix: \begin{...
3
votes
0answers
33 views

Derivative of $\log^2(\det(A))$ w.r.t. matrix A

Given that $A\in\mathbb{R}^{n\times n}$, is there an explicit matrix expression for $$\frac{d}{dA}\log^2(\det(A))$$ (in numerator layout format)? Attempt $$=2\log(\det(A))\frac{d}{dA}\log(\det(A))$...
2
votes
3answers
63 views

Derivative of squared norm of component of a matrix perpendicular to identity matrix, with respect to the original matrix

Let $J\in\mathbb{R}^{n\times n}$ What is the derivative (with respect to $J$) of the squared norm of the component of $J$ that is orthogonal to $I$ (the identity matrix)? Attempt $J$'s projection ...
0
votes
1answer
22 views

trace and derivative: understanding $\text{tr}\left(e_j e_i^T B^T B \right) = \langle B e_i e_j^T, B \rangle $

where $B$ is $n \times n$ matrix Can someone help me understand the above equality. what kind of inner product we are using here? Also, why do we have $\frac{d}{dB}\left(\langle B e_i e_j^T, B \...
2
votes
1answer
32 views

Derivative of L1 norm of Hadamard product

I am trying to find the derivative of $f(B)=\lambda\Vert W \bigodot B \Vert_1 + \frac{\rho}{2}\Vert A-B \Vert_F^2 + tr(\Delta^T(A-B))$ with respect to B. where B is (n×n)matrix, W is (n×n)constant ...
0
votes
1answer
70 views

Least Squares: Derivation of Normal Equations with Chain Rule (Revisited)

My question pertains to someone else's answered question that has made me curious. The OP wanted to differentiate the following using the chain rule: $$ J(\theta)=\frac12(X\theta-y)^T(X\theta - y) $$ ...
0
votes
2answers
59 views

What does $[T]_{\cal BB}$ mean?

I have been given a matrix P where the columns represent a basis in B. I have also been given a matrix A which is the standard matrix for T. I am then supposed to calculate the matrix for T relative ...
1
vote
1answer
58 views

How to write the first term of Taylor expansion for a function matrix?

Assume $F(A)=\left\| A^N v-w \right\|^2$ where $A$ is a square matrix and $v$ and $w$ are two constant vectors and $N$ is a positive integer. Also, the norm is the Euclidean norm. How to write the ...
1
vote
1answer
58 views

Minimizing $\| x A - B\|_F^2$ With a Constraint

I have previously asked an optimization question Here. I will reiterate the question and simply add a constraint to it: I have 2 known grayscale images (256×256 matrices) $A$ and $B$ and want to find ...
1
vote
1answer
45 views

Vector calculus notation

Say we have a $n \times m$ matrix $\boldsymbol{X}$. When we see $\boldsymbol{X}^T_i$, is this typically referring to (1) the i-th row of $\boldsymbol{X}^T$, which is a column vector. So $\boldsymbol{...
2
votes
0answers
53 views

Show that $p^T \Sigma^{-1} p \leq k^2$ is equivalent to $w^Tpp^Tw \leq k^2 w^T \Sigma w$

I am trying to understand a research paper 1 (Eq 11 and Eq 10) which states the solution to the following 2 optimization problems are equivalent. $$\begin{array}{ll} & \min_{p} p^Tw \\ & \...
0
votes
0answers
32 views

How to derive the Lyapunov Equation?

I have a linear dynamical system: $\dfrac{dx}{dt} = Ax$ with $A \in \mathbb{R}^{n\times n}$ and $x \in \mathbb{R}^{n\times1}$. I consider a quadratic function $V(x) = x^T P x$ where $P \in \mathbb{R}...
0
votes
1answer
39 views

Derivative of $f(P^{-1}HP)$ w.r.t. H

Given $f:\mathbb{R}^{n\times n}\rightarrow\mathbb{R}$ $H\in\mathbb{R}^{n\times n}$ is a variable $P\in\mathbb{R}^{n\times n}$ is a constant $f':\mathbb{R}^{n\times n}\rightarrow\mathbb{R}^{n\times n}$...
0
votes
0answers
18 views

Maximize quadratic form under the constraint of a unit hypersphere

I'm trying to maximize $f(\vec{v}) =\vec{v}^TA\vec{v}$ ($A$ is a symmetric matrix, e.g., covariance matrix) under the constraint that $||\vec{v}||^2=\vec{v}^T\vec{v}=1$. I haven't taken any matrix ...
1
vote
1answer
42 views

Derivative of $P^{-1}HP$ w.r.t. $H$

Is there a matrix expression for $\frac{d}{dH}P^{-1}HP$ (for constant square matrix $P$)? Background (if necessary) I seek to design an objective function (and its gradient) over the 8D space of ...
1
vote
1answer
34 views

Taylor-formular for Matrices

Let $A\in\mathbb{C}^{n\times n}$. What does it mean to make a Taylor expansion of $A$ around a point? I assume that the induced linear map is what should be expanded, but I have never seen a Taylor ...
0
votes
0answers
20 views

2 Coin tossed using Markov Chains

A sequence of experiments is executed, in each of the two coins they are thrown. Let $S_1$ indicate that two suns came out, $S_2$ indicate that a sun and an eagle came out, and $S_3$ indicate that two ...
0
votes
1answer
40 views

Interpret a matrix as a function

I have a simple matrix that I don't understand how to write as a function. I have attached a picture of the matrix and have come up with two different answers, A or B. Can anyone give me a hint on ...
1
vote
0answers
40 views

Matrix inverse with diagonal Perturbations

Consider $$ P:= (X^TX+D_1^{-1})^{-1} - (X^TX+D_2^{-1})^{-1} $$ $$ Q:= (X^TX+D_1^{-1})^{-1/2} - (X^TX+D_2^{-1})^{-1/2} $$ $X$ is an $n \times p$ matrix with $p > n$ and $D_1, D_2$ are diagonal ...
0
votes
2answers
42 views

Square matrix 0 x 0?

Can you consider a 0x0 matrix as a square matrix, I can't find the precise definition. I need it for my programming assignment for throwing exceptions.

1
2 3 4 5
52