New answers tagged

1 vote

How to compute the derivative of this matrix equation

This is not a complete derivation, but if you take PhysicsKid's answer and define $$\eqalign{ \def\sym#1{\operatorname{sym}\left(#1\right)} &\sym{X} = \tfrac12\left(X+X^T\right) \qquad&\big({\...
user avatar
  • 27.5k
2 votes

How to compute the derivative of this matrix equation

From the Matrix Cookbook we see that the derivative of an inverse is $$\frac{\partial Y^{-1}}{\partial c} = -Y^{-1}\frac{\partial Y}{\partial c}Y^{-1}$$ We can use that to get $$\frac{d(A^TA)^{-1}}{dc}...
user avatar
3 votes

Why does the matrix exponential $e^A$ always exist?

It is worth to treat the matrix $A$ as a linear bounded operator $\mathcal{L}(\mathbb{R}^n,\mathbb{R}^n)$. Therefore defining the operator $\exp A:\mathbb{R}^n\to \mathbb{R}^n$ as a limit of the ...
user avatar
5 votes

Why does the matrix exponential $e^A$ always exist?

Consider the matrix norm $$\|A\|:=\max_{x\in\mathbb{R}^{n}}\frac{|Ax|}{|x|},$$ where $|\cdot |$ is the Euclidean metric on $\mathbb{R}^{n}$. Let's prove that $\|AB\|\le\|A\|\|B\|$, for all $A,B\in M(n,...
user avatar
  • 1,043
0 votes

Why does the matrix exponential $e^A$ always exist?

If maximal absolute value of element of $A$ is $x$, then maximal absolute value of element of $A^2$ is at most $n \cdot x^2$, of $A^3$ is $n^2 x^3$ and so on - maximal absolute value of element of $A^...
user avatar
  • 7,786
1 vote

How to take the derivative of a matrix with respect to itself?

The answer is a lot easier than the previous posters are indicating $\frac{d}{dX}(A*X)=A$ define $A=I_m$ where $m$ is the number of rows of $X$ because $I_m*X=X$ this is an identity and you get your ...
user avatar
  • 31
1 vote
Accepted

Trying to understand why a certain matrix derivative is sparse

Welcome to MSE :D Simple way to deal with derivatives involving matrix multiplications is to view it via the summation form $$ Y=XW^T\\ Y_{ij}=\sum_k^{1024}X_{ik}W_{jk} $$ So what you mean by $\...
user avatar
0 votes

Trying to understand why a certain matrix derivative is sparse

There are multiple conventions: Let's say $X \in \mathbb R^{a \times b}$ and $W \in \mathbb R^{c \times d}$ for clarity. When you write $\frac{\partial Y}{\partial X}$ then we usually mean the object ...
user avatar
  • 15.7k
2 votes
Accepted

Confusion about convergence for the logarithm of a matrix

Gary supplied the missing link I needed; thank you, Gary! Pick up at the sequence of inequalities in my question: $$||\text{log}A||\le \sum_{m=1}^\infty ||(-1)^{m+1} \frac{(A-I)^m}{m}||=\sum_{m=1}^\...
user avatar
  • 189
2 votes

Matrix Inversion of $f(X) = (X^{-1}-A)$

You could approach this problem using differentials $$\eqalign{ \def\X{X^{-1}} F &= \X - A \\ dF &= -\X\;dX\;\X \\ }$$ Since you want the updated value of the function $F_+$ to equal zero $$dF ...
user avatar
  • 27.5k
4 votes
Accepted

Derivative of the inverse of a symmetric matrix w.r.t itself

$ \def\p{\partial}\def\o{{\tt1}} \def\E{{\cal E}}\def\F{{\cal F}}\def\G{{\cal G}} \def\C{C^{-1}}\def\Ct{C^{-T}} \def\LR#1{\left(#1\right)} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\qiq{\quad\implies\...
user avatar
  • 27.5k
3 votes

Derivative of the inverse of a symmetric matrix w.r.t itself

In the single-variable case, we have that $$\dfrac{d}{dt}C(t)^{-1}=-C(t)^{-1}\dfrac{dC(t)}{dt}C(t)^{-1}.$$ This can obtained by differentiating the expression $C(t)C(t)^{-1}=I$ on both sides with some ...
user avatar
  • 3,125
1 vote

Reverse engineering a matrix in RREF

For example, suppose the RREF is $$ \pmatrix{1 & 0 & 1\cr 0 & 1 & 1\cr 0 & 0 & 0\cr}$$ and your matrix with first two columns known is supposed to be $$ ...
user avatar
0 votes

How to show two matrices are similar?

It is easy to solve the nine linear equations in nine variable for $P$ given by $PA=BP$ with $B={\rm diag}(0,0,3)$, and one obtains several solutions. One invertible $P$ is given by $$ P=\begin{...
user avatar
2 votes
Accepted

Derivative of diagonal matrix expression: $f(X)=\text{diag}(X)M^T\text{diag}^{-1}(MX)$

For typing convenience, define the variables $$\eqalign{ \def\bx{\boxtimes} \def\LR#1{\left(#1\right)} \def\qiq{\quad\implies\quad} \def\A{A^{-1}} \def\o{{\tt1}} \def\p{\partial} \def\grad#1#2{\frac{\...
user avatar
  • 27.5k
1 vote

Derivative of diagonal matrix expression: $f(X)=\text{diag}(X)M^T\text{diag}^{-1}(MX)$

One way to simplify the matter is to use subscripts (einstein summation rule) when dealing with matrix function and derivative. For example your function could be re-writen as such $$ f(X)_{ij}=X_i M_{...
user avatar
0 votes

Differentiating with respect to a vector transpose

The plain simple answer is that the Differentiation of $β^T$($X^T$X)B becomes ($X^T$X)$\hat \beta$ ($\hat \beta$ is the Beta estimate) $B^T$ disappears as usual. `
user avatar
0 votes

Differentiate the summation over an index of a rank 3 tensor.

$ \def\A{{\cal A}}\def\B{{\cal B}}\def\C{{\cal C}} \def\o{{\tt1}}\def\p{\partial} \def\LR#1{\left(#1\right)} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\qiq{\quad\implies\quad} \def\grad#1#2{\frac{\p #...
user avatar
  • 27.5k
3 votes

Column-wise partitioned matrix multiply by Kronecker Sum matrix

So the $\oplus$ in $\lambda \oplus B$ is NOT a Kronecker sum, but a direct sum. This means $\lambda \oplus B$ is essentially a block diagonal matrix with the 1st diagonal block having the size $1 \...
user avatar
  • 51
1 vote
Accepted

What the rank of a matrix with the elements of one column to be infinity?

The following two statements are in direct contradiction: Suppose the $m\times m$ real matrix $A$ if all the elements in the first column and first row are assigned to be infinity Infinity is not a ...
user avatar
  • 114k
1 vote
Accepted

Calculate $\frac{\partial}{\partial \mathbf{A}}\lVert \mathbf{A}^{\top}\mathbf{AX}-\mathbf{X} \rVert _{F}^{2}$

Stripping away the norm, you're attempting a least-squares solution of $$A^TAX = X \quad\implies\quad A^TA = XX^+$$ QR decomposition yields $$\eqalign{ QR &= XX^+ \\ (QR)^TQR &= (XX^+)^T(XX^+) ...
user avatar
  • 27.5k
2 votes
Accepted

Derivative of matrix-diag matrix product

Let $F(x):=W\mathrm{diag}(f(Ax+b))$, then we can simply rewrite it as $$F(x)=W\sum_{j=1}^Ke_je_j^Tf(e_j^TAx+b_j)$$ where $\{e_i\}$ is the natural basis for $\mathbb{R}^K$ and $b=[b_i]$. This is a ...
user avatar
  • 3,125
3 votes
Accepted

Why matrix exponential in two different methods not matching?

It was not true that$$e^{\operatorname{Id}t+Bt+Ct}=e^{\operatorname{Id}t}e^{At}e^{Ct}.$$More generally, it is seldom true that $e^{A_1+A_2}=e^{A_1}e^{A_2}$. The other method is fine.
user avatar
0 votes

Gradient of $A \mapsto \sigma_i (A)$

Here is an answer using some basic convex analysis, and no calculation using coordinates or the canonical basis. It is more useful to work with $$f_r(A) = \sum_{i\le r} \sigma_i(A)$$ (sum of $r$ ...
user avatar
  • 1,080
2 votes
Accepted

Minimizing 2-norm of a matrix

$ \def\a{\alpha}\def\b{\beta} \def\qiq{\quad\implies\quad} $The problem can be broken into two subproblems, each of which has a closed-form for its least-squares solution $$\eqalign{ \a B &= A-c\b^...
user avatar
  • 27.5k
3 votes

Derivative of transpose of matrix

$X↦X^⊤$ is a linear map and so equal to its own derivative. A better question to ask is can we express this function in the standard form of a linear map, which is as a matrix-vector product in the ...
user avatar
  • 9,984
0 votes

Derivative of transpose of matrix

The set of real $n \times n$ matrices forms a real vector space with respect to the usual addition of matrices. Your identification of this space with the canonical vector space $\mathbb{R}^{n^2}$ is ...
user avatar
  • 103
1 vote
Accepted

summation as a matrix multiplication

$ \def\L{{L}}\def\o{{\large\tt1}}\def\p{\partial} \def\LR#1{\left(#1\right)} \def\BR#1{\Bigl(#1\Bigr)} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\Diag#1{\operatorname{Diag}\LR{#1}} \def\Lapl#1{\...
user avatar
  • 27.5k
1 vote

Gradient of Quadratic Function

@Jake, Regarding your question, your computations are all correct. The only thing that is missing is to recognize $$ \mathbf{x}_n^T \mathbf{w} \mathbf{x}_n = (\mathbf{x}_n \mathbf{x}_n^T) \mathbf{w} $$...
user avatar
  • 1,919
2 votes

Integration of matrix form of Vasicek variance (Python/Matlab)

Assuming that $K$ and $\Sigma_x$ are square matrices the explicit solution of your vector SDE is $$ X_t=e^{-Kt}\cdot\Big\{X_0+\int_0^t e^{Ks}\cdot\mu\, ds+\Sigma_x\cdot\int_0^t e^{Ks}\cdot dZ_s\Big\}\,...
user avatar
  • 4,274
1 vote

Problem $2.9.6$ (Perko's ODE): Show $|Y(t)| \le |Y(0)|\exp\left(\int_0^t \|A(s)\|\, ds\right)$ for $Y' = AY$

$ \def\T{{\rm tr}} \def\A{\T(A)} \def\l{\lambda} \def\qiq{\quad\implies\quad} $This well known result $$\eqalign{ &\l = \log|Y| \\ &d\l = \T(Y^{-1}\,dY) \;=\; \T(Y^{-1}AY\,dt) \;=\; \A\;dt \\ &...
user avatar
  • 27.5k
1 vote
Accepted

Problem $2.9.6$ (Perko's ODE): Show $|Y(t)| \le |Y(0)|\exp\left(\int_0^t \|A(s)\|\, ds\right)$ for $Y' = AY$

For $t \ge 0$ we have $Y(t)=Y(0)+ \int_0^t A(s)Y(s)ds$, hence $$ |Y(t)| \le |Y(0)| + \int_0^t \|A(s)\|\cdot|Y(s)|ds. $$ Now (1) follows from the Grönwall-inequality: https://en.wikipedia.org/wiki/Gr%...
user avatar
  • 2,546

Top 50 recent answers are included