Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Filter by
Sorted by
Tagged with
0
votes
1answer
33 views

Understanding a specific process of finding the derivative of $x^TAx$

I am referring to @copper.hat's response to : Derivative of Quadratic Form. I do not have the reputation to reply directly. My goal is to find a way to better differentiate and understand these ...
0
votes
0answers
12 views

The normal equation for Multivariate Multiple linear regression

I am confused about the normal equation for Multivariate Multiple linear regression. I don't know How could I derive the normal equation for this case. Consider the multivariate multiple linear ...
1
vote
0answers
34 views

Minimizing trace involving inverse

Given symmetric positive semidefinite rank-$r$ matrix $R \in \mathbb{C}^{m \times m}$, where $r < m$, and scalar $p \geq 0$, I have the following optimization problem in matrix $X \in \mathbb{C}^{n ...
-2
votes
1answer
32 views

What if the sum of the elements of the inverse of a matrix is 0

If the sum of the elements of an inverse of a matrix $A$ is $0$, does it affect somehow the determinant or something else of the matrix $A$, where $A$ is a $3\times 3$ matrix? I tried for a $2\times 2$...
3
votes
1answer
48 views

Writing Back-Propogation out in terms of Matrix Multiplication

I am trying to write my own backpropogation algorithm for neural networks for a class. For any specific weight of my network I could easily take the derivative, but for computational speed I want to ...
1
vote
2answers
32 views

Matrices and images [closed]

Let T: $\mathbb{R}^4 \to \mathbb{R}^5$ defined by $T (x_1, x_2, x_3, x_4) = (x_4, x_1, x_3, x_2, x_1 - x_3)$ A) Find the standard matrix for the linear transformation $T$ B) Find the image of $x = (1, ...
2
votes
1answer
34 views

Derivative of trace involving Hadamard product

Let us assume that $A, S\in\mathbb{R}^{n\times n}$, $U\in\mathbb{R}^{n\times k}$, and $V\in\mathbb{R}^{n\times k}$. I am trying to differentiate the following expression: $$\Phi(U,V)=\mathrm{trace}\...
0
votes
0answers
23 views

Full derivative of a scalar function wrt vector - chain rule

This confusion arose from this matrix calculus helper, the chart of total derivatives on page 23. The case with the scalar function of a vector of $n$ variables: it can be written as a product of a $1\...
1
vote
1answer
70 views

How to derive differential gradient descent from cost function using matrix calculus?

I am new to matrix calculus and I hope you can help me with this question! I’m trying to derive the update law for gradient descent to minimize a cost function $$j(\mathbf{X}) = \frac{1}{2} \mathbf{e}^...
0
votes
2answers
75 views

Differentiate vector transpose using rules [duplicate]

I am referring to Tom Minka's Old and New Matrix Algebra Useful for Statistics. I don't have the book by Magnus & Neudecker so I can't refer to the details of the theory. Regarding rules (6): $d(...
1
vote
1answer
39 views

Calculating the partial derivative with respect to a matrix

Say, $y = W^T b$ where, $ W^T \in R^{2 \times 2} $ and $ b \in R^{2 \times 1} $. Now, we want to calculate: $\frac{\partial y}{\partial W}$. First, if we look at the dimensions: $y \in R^{2 \times 1}$....
1
vote
1answer
35 views

Computing the Jacobian $J_F$ with $F = h \circ f$

Let $f : \mathbb{R}^l \rightarrow{} \mathbb{R}^m$ and $h : \mathbb{R}^m \rightarrow{} \mathbb{R}^o$ and let $F = h \circ f$ with $F : \mathbb{R}^l \rightarrow{} \mathbb{R}^o$ I want to ...
4
votes
2answers
78 views

Find minimum of $f(\mathbf{v})= \mathbf{v}^\top A \mathbf{v} - \sum_{i=1}^N \log((\mathbf{v}^\top \mathbf{u}_i)^2) $

Consider the function: $$ f(\mathbf{v})= \mathbf{v}^\top A \mathbf{v} - \log((\mathbf{v}^\top \mathbf{u})^2), $$ where $\mathbf{v}, \mathbf{u} \in \mathbb{R}^n$ and $A \in \mathbb{R}^{n \times n}$ is ...
0
votes
0answers
32 views

Graph Theory Question regarding Isomorphism and Matrix Transposes

How would one prove true or false the statement "for an I, incidence matrix of a graph G, and a J, incidence matrix of a graph H, it is impossible for G and H to be isomorphic AND for the product ...
1
vote
0answers
42 views

matrix algebra problem

For every $ n> 2, n\epsilon \mathbb{N}$ let A,B,C,D be matices $ A,B,C,D\epsilon M_{n}(\mathbb{R})$ such that $ AC-BD=I_{n}$ and $AD+BC=0_{n} $. Prove that: a) $CA-DB=I_{n}$ and $DA+CB=0_{n}$ b)$ ...
0
votes
1answer
32 views

Solution for nonhomogenous linear matrix ODE

For matrices $X, A, B\in \mathbb{R}^{n\times n}$, $X = X^\top, B=B^\top$, I am looking to find an explicit solution to the following linear first-order matrix ODE where $A$ and $B$ are constant; $ \...
0
votes
2answers
38 views

matrix for this system of non-linear equation?

Let's consider the product of a square matrix and a vector: $ M{\times}v = \begin{bmatrix} 1 & 2 & 3\\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix} {\times} \begin{bmatrix} a \\ b \\ c ...
0
votes
1answer
52 views

Does there exist a way to compute matrix “properties” (eigenvalues, determinant, etc.) from its vectorized representation?

I am wondering whether there are some formulas that compute matrix eigenvalues, determinant, rank, etc. directly from its vectorized representation. More formally: suppose $\bf S \in \mathbb{S}^N$, i....
0
votes
0answers
30 views

Power series expansion for a circulant matrix

Let $A$ the block matrix given by the blocks: $$\tilde{A}=\begin{pmatrix} 1&-\mu&0&...&0&-\mu\\ -\mu&1&-\mu&...&0&0\\ 0&-\mu&1&...&0&0\\ ...&...
0
votes
1answer
42 views

Derivative of $\exp[(aX+Y)t]$ w.r.t $a$ where $X,Y$ are matrices of dimension $n\times n$

Let $X,Y\in\mathbb{R}^{n\times n}$ and define $$ F(a,t) = \exp[(aX+Y)t]. $$ Then, compute $$ \frac{\partial F}{\partial a} = ? $$ My attempt: Let $M_a = aX + Y$, then $$ \frac{\partial F}{\partial a} =...
3
votes
0answers
72 views

KKT conditions for $\max \log \det(X)$ with LMI constraints

I am trying to derive the KKT conditions for the following convex optimization problem where $A$ is a given matrix: $$\begin{array}{ll} \underset{X,Y,Z}{\text{minimize}} & - \log \det \left(I + Z +...
0
votes
1answer
49 views

Find the gradient (with respect to a matrix) of an expression containing a Frobenius norm and a Hadamard product.

I'm struggling with taking the gradient with respect to (w.r.t) the matrices $H_R$ and $H_I$ in the following expression $$\left\| Z - I\odot(H_R^TA + H_I^TB) \right\|_F^2 + \left\|W-\begin{pmatrix} ...
3
votes
0answers
40 views

Derivative of $(A+BC^{-T}B^T)^{-1}BC^{-1}$

Suppose $A:\mathbb{R}\rightarrow\mathbb{R}^{m\times m}$, $B:\mathbb{R}\rightarrow\mathbb{R}^{m\times n}$, and $C:\mathbb{R}\rightarrow\mathbb{R}^{n\times n}$. I'm trying to find the derivative with ...
4
votes
0answers
116 views

How to show that a matrix is not full rank using the linear independence definition

I want to show that the matrix $A$ defined below is NOT full column rank. I have a proof but it does not look complete to me. Could you help to fix it? Consider a matrix $A$ structured as follows $$ A=...
0
votes
1answer
45 views

Matrix norm differentiation problem.

Consider $J = \|X - AS\|^2$, where $A \in \mathbb{R}^{N \times R}$, $S \in \mathbb{R}^{R \times M}$. We need to calculate $\dfrac{d J}{d S}$. I've done the following: $J = \operatorname{tr}((X - AS)(X ...
1
vote
0answers
30 views

Computational complexity of the Cholesky factorization

According to the Cholesky factorization on Wikipedia, the computational complexity of it is $\frac{n^3}{3}$ FLOPs where $n$ is the size of the considered matrix $\mathbf{A}$. There are various ...
0
votes
1answer
39 views

Square root of a block matrix

Given the $2n \times 2n $ matrix $$A=\begin{pmatrix} \bar{A} & A^* \\ A^* & \bar{A} \end{pmatrix}$$ let $X$ the matrix such that $XX=A$, that is $X=\sqrt{A}.$ So $$X=\begin{pmatrix} X_{1,1}&...
0
votes
0answers
26 views

Square root of a circulant block matrix

I'm trying to show the following: Given the following $n\times n$ symmetric circulant matrices $$A^*=\begin{pmatrix} 1 & -\mu_a & 0 & ...&0&-\mu_a \\ -\mu_a & 1 & -\mu_a &...
1
vote
1answer
35 views

Is trace( inv(A) B ) a distance?

Given two symmetric p.d. matrices of dimension $(p \times p)$, $\mathbf{A}$ and $\mathbf{B}$, can we interpret $$ d = \operatorname{trace}(\mathbf{A}^{-1} \mathbf{B}) $$ as a distance between the two? ...
6
votes
2answers
88 views

Matrix equation with diagonal matrix

Let $A \in\mathbb{R}^{n\times n}$ be a given Positive-Semi Definite matrix ,$\theta\in\mathbb{R}^n$ a vector, $x\in\mathbb{R}$ an unknown variable, and $a\in\mathbb{R}^+$ a positive constant. I have ...
0
votes
1answer
47 views

conjugate function of log trace of matrix exponential

Given an n-by-n real symmetric matrix X, define $$e^X = \sum_{k=0}^{n}\frac{X^k}{k!}$$ Derive the Fenchel conjugate of $$f(X) = log(Tr(e^X))$$ as a function on n-by-n real symmetric matrices. Here Tr ...
1
vote
0answers
29 views

How to derive the linearized form of the PDE the “singular values are constant”?

$\newcommand{\Cof}{\operatorname{Cof}}$ Let $0<\sigma_1<\sigma_2<\cdots<\sigma_n$ and set $A=\operatorname{diag}(\sigma_1,\dots,\sigma_n)$. Claim: Let $B $ be a real-valued $n \times n$ ...
3
votes
2answers
28 views

Good definition for $\int_a^\infty X(t)dt$, where $X(t)\in M_n$

Any comments on this? : I'm reading a book, and some exercise says: Provide a good definition for $\int_a^\infty X(t)dt$, where $X(t)\in M_n(\mathbb{R})$ and prove that if $$\int_a^\infty \|X(t)\|dt \ ...
2
votes
1answer
20 views

Equivalence $\|A\|=\inf\left\lbrace c>0:\|Ax\|\leq c\|x\| \right\rbrace$

Prove that if $A$ is a matrix, and we defined $$\|A\|=\sup\left\lbrace\frac{\|Ax\|}{\|x\|}:x\neq 0\right\rbrace $$ then $$\|A\|=\inf\left\lbrace c>0:\|Ax\|\leq c\|x\| \right\rbrace $$ Attempt: $\|A\...
2
votes
2answers
98 views

Implicitly Differentiating a System of Matrix Equations

Setup Let $\mathbf a$ be an arbitrary $m\times 1$ vector, $\mathbf B$ be an arbitrary $n\times m$ matrix, with $m>n$, and $\mathbf C$ be a symmetric $m\times m$ matrix. The scalar $\lambda$ is also ...
1
vote
1answer
48 views

Loss function gradiant - why the transpose?

I have a product of two matrices, defined as $Y=AB$. I need to find $\nabla_A J$ where $J(Y)$ is a scalar-valued loss function dependent on $Y$. As I understand, this gradient can be written as $\...
1
vote
1answer
39 views

Derivative Matrix with general function and product

Given $F,f:\mathbb R\to\mathbb R$ such that $F'=f$ and $\pmb a,\pmb b\in\mathbb R^n$, compute $$\frac{d}{d\pmb X}\left(\pmb a^T F\left(\pmb X\right)\pmb b\right)$$ where $\pmb X\in\mathbb R^{n\times n}...
2
votes
0answers
75 views

Why is there a difference between MATLAB and manual calculation for eigenvectors

We have a matrix $A$ as below $$\begin{bmatrix}1&2&2\\0&1&2\\0&0&2\end{bmatrix}.$$ I know the eigenvalues are 1 and 2 because matrix $A$ is a triangular matrix. And by working ...
1
vote
0answers
38 views

Spectral norm of $(1/(i-j))$ is smaller than $\pi$

Could someone give me a hint on how to prove that the spectral norm of an $n×n$ matrix $A = (a_{ij})$, where $a_{ii} = 0$ and $a_{ij} =1/(i-j)$, is smaller than $\pi$. $$B=\begin{bmatrix}0 & -1 &...
1
vote
1answer
25 views

Applying the chain rule in matrix form to prove Loss function's derivatives…

I want to prove $\nabla_A J = \nabla_Z J \cdot B^T$ where $Z=AB$. $A$ is a $m \times n$ matrix and $B$ is a $n \times k$ matrix. The function $J$ is not given to me. I began this proof by first ...
0
votes
1answer
40 views

How to take the derivative of $ y = xA^T + b$ w.r.t. $A$ and $b$, where $x,b$ are vectors and $A$ is a matrix

How exactly could I take the derivative of the following expression? $$ y = xA^T + b$$ Let's say that I have $x \in \mathbb{R}^{n}$, $A \in \mathbb{R}^{m,n}$, $y \in \mathbb{R}^{m}$, and $b \in \...
0
votes
1answer
22 views

Derivative of trace involving hadamard product and product of inverse matrices

I need to find the derivative with respect to $\mathbf{\Omega}$ of $$ Tr\left(\left(\left(\mathbf{\Omega^{-1}}\mathbf{C}\mathbf{\Omega^{-1}}\right)\circ\mathbf{I}\right)\mathbf{S}\right) $$ In the ...
2
votes
1answer
41 views

Calculate the gradient and Hessian of $x_0^T(X\operatorname{Diag}(w)X^T)^{-1}x_0$ w.r.t. vector $w$ in matrix calculus?

I'm trying to calculate the Hessian matrix of the formula $$ f(w) = x_0^T(XWX^T)^{-1}x_0 $$ where $w=(w_1,\cdots ,w_n)^T$, $W=\operatorname{Diag}(w)$ is a diagonal matrix that has $w$ as diagonal ...
2
votes
1answer
30 views

Derivative of submatrix with respect to the whole block matrix

I am reading a research paper and getting stucked with how they derived a formula. Suppose that we have the following block matrix $$\underset{d \times d}{\boldsymbol{A}} = \begin{bmatrix} \underset{q ...
1
vote
2answers
49 views

Change from differentiation wrt to matrix to wrt to inverse of matrix for symmetric matrices

For the rule below: $$ \frac{\partial J}{\partial \mathbf{A}}= -\mathbf{A}^{-T} \frac{\partial J}{\partial \mathbf{W}} \mathbf{A}^{-T} $$ where $\mathbf{A}$ is an invertible square matrix, $\mathbf{W}...
2
votes
1answer
56 views

Is the support of $\rho_{AB}$ always contained in the support of $\rho_A\otimes\rho_B$?

Given a positive unit trace Hermitian matrix, i.e. a density operator, $\rho_{AB}$ on Hilbert spaces $\mathcal{H}_A\otimes\mathcal{H}_B$. Consider its marginals $\rho_A$, $\rho_B$. Do we have the ...
3
votes
4answers
66 views

Matrix Derivative Product Rule $\frac{d}{dx} \langle x,y\rangle Ax $

Let $A\in\mathbb{R^{n\times n}}$,$x,y\in\mathbb{R^{n}}$. What is the derivative of the following expression? $$ \frac{d}{dx} \langle x,y\rangle Ax $$ What I tried: Applying the product rule doesn't ...
0
votes
0answers
36 views

Inequalities for trace of a product

For positive-definite matrices $A$ and $B$ the following inequalities hold: $$ 0\leq\lambda_{\min}(A)\lambda_{\max}(B)\leq\lambda_{\min}(A){\rm Tr}(B)\leq{\rm Tr} (AB)\leq\lambda_{\max}(A){\rm Tr} B\...
-2
votes
2answers
36 views

How may random unique combinations a 4x4 matrix can make

Let's say we have a matrix (grid) of 4x4 that has values from 0 to 15 in it. 0 1 2 3 a = 4 5 6 7 8 9 10 11 12 13 14 15 If this array is ...
0
votes
0answers
17 views

Lower estimation for trace of a product

Let $X\succ0$ and $Q$ are square real matrices. For $Q\succ0$ we have the estimation $$ {\rm Tr}(XQ)\succeq{\rm Tr}(X)\lambda_{\min}(Q). $$ Is it possible to obtain the similar lower estimation for (...

1
2 3 4 5
60