Questions tagged [matrix-calculus]
Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.
1
vote
0answers
15 views
Gradient of the Lie exponential map on SO(n)
I am interested in computing the gradient of $f(e^A)$ when $A$ is a skew-symmetric matrix.
If we write $e^A = B$ and we denote the gradient of the function $f$ on the ambient space evaluated at $B$ ...
-2
votes
0answers
16 views
1
vote
0answers
21 views
matrix block multiplication definition, properties and applications
I would like to have a clear definition of matrice block multiplication, its properties and some applications. If possible, some book references.
Suppose we have $A B $ , where $A$ and $B $ are ...
1
vote
0answers
17 views
Finding the product of two normal distributions
Using the formulas given in section 8.1.8 of this handbook, what is going to be the result of following integral
\begin{equation}
\int\mathcal{N}_{\mathbf{A}\mathbf{X}}(\mathbf{m}_1,\Sigma_1)\mathcal{...
2
votes
0answers
89 views
Matrix functions reducing eigenspectrum degeneracy?
Standard matrix functions fulfill $spec(f(A))=f(spec(A))$ relation for eigenspectrum, maintaining degeneracy if $f$ is injection. For simplicity we can focus on real symmetric matrices.
However, ...
1
vote
0answers
27 views
What can I say about $\mathbf{X}$ if $\mathbf{y}^{\mathrm{T}} \mathbf{X} \mathbf{y} = \mathbf{z}^{\mathrm{T}} \mathbf{X} \mathbf{z}$?
Consider the $N$-dimensional positive semidefinite matrix $\mathbf{X}$ and two $N$-dimensional vectors $\mathbf{y}$ and $\mathbf{z}$ with $\|\mathbf{y} \|^{2} = \| \mathbf{z} \|^{2}$. If I know that $\...
0
votes
1answer
56 views
Write the derivative of the lower-triangular matrix $L(t)$ in terms of $L(t)$, $L^{-1}(t)$ and $\frac{d}{dt}A(t)$, where $A(t)=L(t)L^T(t)$
Let $A(t)$ be a symmetric positive definite matrix, thus by Cholesky decomposition, we have $A(t)=L(t)L^T(t)$ where $L(t)$ is lower triangular. Suppose $A(t)$ is differentiable.
I want to write $\...
0
votes
0answers
20 views
Derivative of Hadamard product with respect to matrix
I'm trying to calculate this derivative wrt matrix $F_{i}$ and simplify the whole expression:
$ \frac{ \partial J_{term 1}}{\partial \mathbf{F_{i}}}=
2 (\sum_{j:G(i,j)=1} (\mathbf{W}_{i,j} \...
-1
votes
0answers
12 views
How to solve this matrix orthogonal problem? [on hold]
The problem is $Q^*=\arg\min \limits_{Q}Tr(Q^TAQ) +\frac{\mu}{2}||Q^TX-E ||_F^2\ \ s.t. Q^TQ=I$ ,where $Tr$ is matrix trace operator.
Thanks.
1
vote
2answers
29 views
Why is $\triangledown_x x^TA^TAx = 2A^TAx$ not $2x^TA^TA$?
A proof I'm looking at shows $\triangledown_x x^TA^TAx = 2A^TAx$.
I did Matlab symbolic calculation to verify this, but I found the converse. It should be $2x^TA^TA$.
$\frac{d}{dx}x^TA^TAx = \\ [ ...
0
votes
1answer
33 views
How to determine basis and dimension of subspaces?
I'm new into linear algebra and I encountered the following problem:
In $\mathbb{R}^4$ we conside the following subspaces: $U=\{(x.y.z.w)\in \mathbb{R}^4 | x+y+z=0\}$, $V=\mathscr{L}((1,1,0,0), (2,-...
1
vote
1answer
23 views
Calculate variations of eigenvectors w.r.t input matrix
From the eigen decomposition :
$$A = PDP^T$$
I would like to calculate $dP$ w.r.t $dA$.
I start like this :
$$dA = dPDP^T + PdDP^T + PDdP^T$$
Since we only consider variations in eigenvectors and ...
0
votes
0answers
23 views
Question on matrix representation of equations
The question is:
In a flock of wild Hoatzins, the females can be classified as being either chicks (up to 1 year old) or adults.
Each year, for every 100 adult females, 50 female chicks are born. ...
0
votes
1answer
20 views
How to get 4x4 matrix from integrating column vector and its transpose?
This is from Finite Element Methods. Part of potential energy approach. Can someone show me how to come to this symetric matrix:
$$ [K]=EI/L^3\begin{bmatrix}
12&6L&-12&6L\\
&4L^2&-...
1
vote
0answers
24 views
Minimization Problem involving PCA
Starting from this equation to estimate:
$\DeclareMathOperator{\tr}{Trace}$
$$ X = F^k \Lambda' ^ {k} + e, $$
with $ X$ an $T\times N $ matrix, $ F$ is $T\times k $ and $ \Lambda$ is $k\times k $, ...
5
votes
1answer
45 views
Points of continuity in $ M_n (k) $ of minimal polynomials
The goal is to show that $\Gamma $ the set of points of continuity of $ M \mapsto \pi_M $ is $ \{ M \in M_n(k) , \chi_M = \pi_M \} $ . Where $ \pi_M $ is the minimal polynomial of $M$ and $ \chi_M $ ...
0
votes
2answers
28 views
It can happen that the norm 1 of a matrix and the infinite norm are different?
I have practiced some exercises with these two norms and in all of them I had the same result, until I tried with
$\begin{pmatrix}
5 & -3 & 2 \\
4 & 8 & -4 \\
2 & 6 & -1 \\
\...
0
votes
1answer
44 views
What is the name of this type of matrix multiplication?
I don't know the name of this type of matrix multiplication. In this type, we multiply each same indexes of matrices and generate our new matrix. Please help me find out.
Normal matrix multiplication ...
1
vote
1answer
16 views
Summation of polynomial matrix multiplication in terms of vector outer product
Consider the following summation
$$
\sum_{i=1}^{T-1}C(A^i-A^{i-1})Bx_{t-i}
$$
where $A$ is a $d \times d$ diagonal matrix, i.e. $A=\text{diag}(\alpha_1,\cdots,\alpha_d)$, $C$ is an $m \times d$, $B$ ...
-1
votes
0answers
17 views
Aligning point cloud using camera local coordinates only [closed]
I have the stanford bunny point clouds taken from different views and the config file containing all the local camera poses for each of these clouds.
...
2
votes
1answer
36 views
gradient and hessian of $e^{x^Tx}$
I want to find gradient and hessian of $e^{x^Tx}$
My attempt: $\nabla = 2x^Te^{x^Tx}$
Hessian $= 2e^{x^Tx}I + 2xx^Te^{x^Tx}$
Is that correct?
0
votes
1answer
33 views
How does the $Tr(nX^{n-1})$ become $n(X^{n-1})^T$?
For a unstructured square complex matrix X. Show that
$\frac{\partial Tr(X^n)}{\partial X}=n(X^{n-1})^T $ and $\frac{\partial Tr(X^n)}{\partial X^*}=n(X^{n-1})^T$.
I think $\frac{\partial Tr(X^n)}{...
0
votes
0answers
29 views
Gradient and hessian of $\log(1+x^TAx)$
I want to find Gradient and hessian of $log(1+x^TAx)$ where $A$ is positive semi definite.
My attempt: $\nabla = x^T(A+A^T)/1+x^TAx$
Hessian = $(A+A^T)(1+x^TAx)-(x^T(A+A^T))^Tx^T(A+A^T)/(1+x^TAx)^2$
...
2
votes
1answer
54 views
Why is $\frac{\partial \operatorname{Tr}(\mathbf X^n)}{\partial \mathbf X}=n\left(\mathbf X^{n-1}\right)^T ?$
Why is $\frac{\partial \operatorname{Tr}(\mathbf X^n)}{\partial \mathbf X}=n\left(\mathbf X^{n-1}\right)^T ?$ and why is $\frac{\partial (\ln[\det(\mathbf X)])}{\partial \mathbf X}=\mathbf X^{-T} ?$
...
1
vote
0answers
23 views
Derivative of a scalar given by a function times its hessian
Is there any way of doing the below but by avoiding $d^3f\over dx^3$?
If for a row vector $x$ e.g. size (3x1):
$f=f(x)=scalar$
$g={df\over dx}$ i.e. the gradient
$H={d^2f\over dx^2}$ i.e. the ...
1
vote
0answers
44 views
Continuity of matrix trace functionals
Let $P$ be a positive $n\times n$ matrix over $\mathbb{C}$, let $f:(0,\mathbb{R})\rightarrow\mathbb{R}$ be a continuous function, and define a function on positive definite $n\times n$ matrices as
$$
...
0
votes
2answers
36 views
Derivatives Across Summations
So, I took one intro course in Tensor calculus and this problem reminds of that, except I can't quite recall how derivatives work with respect to components, or what those derivatives produce. ...
3
votes
1answer
38 views
Eigenvalues of the Product of a Diagonal and a Symmetric Matrix
Let $A \in \mathbb{R}^{n\times n}$ be a symmetric matrix and $D \in \mathbb{R}^{n\times n}$ be a diagonal matrix with positive entries. Prove that the matrix $P:=DA$ has real eigenvalues.
2
votes
2answers
51 views
Derivative of a vector times its transpose
I am trying to work out how to solve a derivative of the form:
$${d \over dx}(M(x)M(x)^T)$$
where M is a vector.
In my case specifically, M is the (1x3) vector
$$M(x)={df(x) \over dx}$$
where f(...
0
votes
1answer
44 views
Prove: $\nabla_A \text{tr}\left(ABA^TC\right) = CAB + C^TAB^T$
Edit: there's another post asking the same thing, but it is not satisfactorily answered. At least not in what I believe is close to the simplest way.
Trying to prove property 3 below, in a "clean" ...
1
vote
1answer
30 views
Derivative of $\text{trace}(U^T x y^T V)$ with respect to $x$
I'm trying to compute the derivative of $\text{trace}(U^T x y^T V)$ with respect to $x$, where $U \in \mathbb{R}^{d_x \times k}$, $V \in \mathbb{R}^{d_y \times k}$, $x \in \mathbb{R}^{d_x}$, and $y \...
2
votes
1answer
22 views
Necessary conditions for convergence of matrix series [closed]
I am interested in necessary conditions under which the series
$$\sum_{n=0}^\infty A^n \Omega (A^n)^{\top}$$
converges wrt. a submultiplicative matrix norm, where A is diagonalizable and $\Omega$ is ...
0
votes
1answer
41 views
Search for a constant Metric $\eta$ satisfying the a condition
I am looking for a $2 \times 2$ constant metric $\eta$, satisfying the following condition:
$\eta A \eta^{-1}= A^T$.
where
$$A = \begin{bmatrix}
a & b \\
c & d
\end{bmatrix}$$
...
4
votes
1answer
70 views
Differential of determinant in the Itô calculus
I'm trying to answer a question about a particular Itô equation $$\mathrm{d}X_t = AX_t\mathrm{d}t+X_tB(\mathrm{d}W_t)$$ where $X_t$ and $A$ are $n\times n$ square matrices and $B : M_n\to M_n$ ($M_n$ ...
3
votes
1answer
129 views
Directional derivatives of matrix trace functionals
Let $P$ be an $n\times n $ positive semidefinite matrix over $\mathbb{C}$, let $p\in\mathbb{R}$ be in the range $0<p<1$. Consider the function $g:[0,\infty)\rightarrow\mathbb{R}$ defined by $g(x)...
1
vote
0answers
29 views
Singularity of a block matrix
When is the matrix $\begin{bmatrix}A & B \\C & D \end{bmatrix}$ singular? I've found on the internet that if $A,B,C,$ or $D$ is non-singular and its Schur complements is non-singular then the ...
1
vote
2answers
50 views
Does this matrix identity hold? (Left and right-product with diagonal matrix)
I have been attempting to prove the following identity:
$\frac{\partial [W^{\frac{1}{2}} K W^{\frac{1}{2}}]}{\partial \hat{f_i}} = K \frac{\partial W}{\partial \hat{f_i}}$
where $W^{\frac{1}{2}}$ is ...
-1
votes
1answer
37 views
Derivative of a products with matrix exponential
I should find the derivative with respect to the vector $a \in \mathbb{R}^{n}$
$\dfrac{\partial}{\partial a}(a^{T}exp(aa^{T})a)$
Answer should be in a matrix form
I tried to decompose the ...
1
vote
1answer
65 views
Matrix differentiation: $\frac{\partial{w}}{\alpha}$ for $w=(X^\top X + \alpha \boldsymbol{I})^{-1}X^\top y$
What is $\frac{\partial{w}}{\partial{\alpha}}$ for $w=(X^\top X + \alpha \boldsymbol{I})^{-1} X^\top y$ where X is an $N \times D$ matrix, y is an N dimensional vector, $\boldsymbol{I}$ is an identity ...
0
votes
0answers
34 views
General chain rule - transpose
There is a function:
$$g_{(x)}:\mathbb{R}^n \rightarrow \mathbb{R}$$
The $1_{st}$ derivative is gradient, $\mathbb{R}^{n}$ column vector. Second derivative is Hessian, $\mathbb{R}^{n*n}$ matrix.
...
2
votes
1answer
69 views
Trace of a Matrix: when to use? what is trace trick?
On calculating log-likelihood function for some multivariate distributions, such as multivariate Normal, I see some examples where the matrices are suddenly changed to trace, even when the matrix is ...
0
votes
1answer
27 views
Unsure of how to proceed through Trace Property
I'm having difficulty figuring out how to derive the following from Andrew Ng's CS229 lecture notes.
$$\nabla_A \textrm{Tr } ABA^{T}C = CAB + C^TAB^T $$
where $\textrm{Tr }$ is the trace operator ...
0
votes
1answer
38 views
Simplify sum of quadratic forms to have matrix variable $\sum_i x_i^T A_i x_i$?
Let $A_i$ be a square symmetric matrix of size $n \times n$ and $x_i$ be the $i$-th row of matrix $X$ of size $m \times n$.
Consider $f(x_1, x_2,...,x_m) = \sum_{i=1}^m x_i^T A_i x_i $.
I need to ...
1
vote
0answers
28 views
Trace of the square root inside commutation property
Hi I'am looking forward to prove this statement, which seems to be true (checked numerically) even if I cannot find it anywhere and didn't manage to prove it :
$\forall B\in M_{n,r}(\mathbb{R}),Tr(\...
1
vote
1answer
46 views
What are the roots in this function.
I am trying to find roots for a particular function. It has reduced to the following expression.
$$
\frac{d f(\lambda)}{d \lambda}=8\lambda + 2\text{Trace}(Q\Sigma Q^\top)- \sum_i \frac{2 M_i}{1- 2\...
0
votes
1answer
33 views
Linear Matrix differential equation
Let $A,B$ be non-singular matrices of dimension $n\times n$. Is there a way to solve the differential equation
$$
f(x)Bx=A\nabla_x f(x)?
$$
I've looked in many places and it doesn't seem to be ...
1
vote
0answers
49 views
Upper bound for $(AB-BA)x$
Given matrices $A,B\in\mathbb{R}^{n\times n}$where matrix $A$ is a diagonal matrix and $B$ is an upper triangular matrix. I'm looking for an upper bound for the expression
\begin{align*}
(AB-BA),
\end{...
0
votes
1answer
41 views
Derivative of multivariate normal density
I am not too familiar with matrix calculus, how can we go about taking the derivative of this quantity:
$$
\nabla_{x}Ax\operatorname{det}(2\pi\boldsymbol\Sigma)^{-\frac{1}{2}} \, e^{ -\frac{1}{2}(\...
1
vote
3answers
63 views
Eigenvalues and eigenvectors of $4\times 4$ matrix
Let M the $4\times 4$ matrix:
$M = \begin{pmatrix}
0 &-2 & 4 &-2 \\
1 &1 &-2 &-1 \\
0 &0 &0 &0 \\
1 &-1 &2 &-3
\end{pmatrix} $ and $M^n = (...
2
votes
1answer
62 views
Is there a closed formula for the positive-definite matrix square root?
Let $S_{>0}$ be the space of symmetric positive definite real $n \times n$ matrices.
Is there a closed-form formula for the positive square root function $\sqrt \cdot :S_{>0} \to S_{>0}$?
...
