5 votes

Why does the matrix exponential $e^A$ always exist?

Consider the matrix norm $$\|A\|:=\max_{x\in\mathbb{R}^{n}}\frac{|Ax|}{|x|},$$ where $|\cdot |$ is the Euclidean metric on $\mathbb{R}^{n}$. Let's prove that $\|AB\|\le\|A\|\|B\|$, for all $A,B\in M(n,...
user avatar
  • 984
4 votes
Accepted

Derivative of the inverse of a symmetric matrix w.r.t itself

$ \def\p{\partial}\def\o{{\tt1}} \def\E{{\cal E}}\def\F{{\cal F}}\def\G{{\cal G}} \def\C{C^{-1}}\def\Ct{C^{-T}} \def\LR#1{\left(#1\right)} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\qiq{\quad\implies\...
user avatar
  • 27.5k
3 votes

Why does the matrix exponential $e^A$ always exist?

It is worth to treat the matrix $A$ as a linear bounded operator $\mathcal{L}(\mathbb{R}^n,\mathbb{R}^n)$. Therefore defining the operator $\exp A:\mathbb{R}^n\to \mathbb{R}^n$ as a limit of the ...
user avatar
3 votes

Derivative of the inverse of a symmetric matrix w.r.t itself

In the single-variable case, we have that $$\dfrac{d}{dt}C(t)^{-1}=-C(t)^{-1}\dfrac{dC(t)}{dt}C(t)^{-1}.$$ This can obtained by differentiating the expression $C(t)C(t)^{-1}=I$ on both sides with some ...
user avatar
  • 3,100
3 votes

Column-wise partitioned matrix multiply by Kronecker Sum matrix

So the $\oplus$ in $\lambda \oplus B$ is NOT a Kronecker sum, but a direct sum. This means $\lambda \oplus B$ is essentially a block diagonal matrix with the 1st diagonal block having the size $1 \...
user avatar
  • 51
3 votes
Accepted

Why matrix exponential in two different methods not matching?

It was not true that$$e^{\operatorname{Id}t+Bt+Ct}=e^{\operatorname{Id}t}e^{At}e^{Ct}.$$More generally, it is seldom true that $e^{A_1+A_2}=e^{A_1}e^{A_2}$. The other method is fine.
user avatar
3 votes

Derivative of transpose of matrix

$X↦X^⊤$ is a linear map and so equal to its own derivative. A better question to ask is can we express this function in the standard form of a linear map, which is as a matrix-vector product in the ...
user avatar
  • 9,984
2 votes

Integration of matrix form of Vasicek variance (Python/Matlab)

Assuming that $K$ and $\Sigma_x$ are square matrices the explicit solution of your vector SDE is $$ X_t=e^{-Kt}\cdot\Big\{X_0+\int_0^t e^{Ks}\cdot\mu\, ds+\Sigma_x\cdot\int_0^t e^{Ks}\cdot dZ_s\Big\}\,...
user avatar
  • 4,254
2 votes
Accepted

Minimizing 2-norm of a matrix

$ \def\a{\alpha}\def\b{\beta} \def\qiq{\quad\implies\quad} $The problem can be broken into two subproblems, each of which has a closed-form for its least-squares solution $$\eqalign{ \a B &= A-c\b^...
user avatar
  • 27.5k
2 votes
Accepted

Derivative of diagonal matrix expression: $f(X)=\text{diag}(X)M^T\text{diag}^{-1}(MX)$

For typing convenience, define the variables $$\eqalign{ \def\bx{\boxtimes} \def\LR#1{\left(#1\right)} \def\qiq{\quad\implies\quad} \def\A{A^{-1}} \def\o{{\tt1}} \def\p{\partial} \def\grad#1#2{\frac{\...
user avatar
  • 27.5k
2 votes
Accepted

Derivative of matrix-diag matrix product

Let $F(x):=W\mathrm{diag}(f(Ax+b))$, then we can simply rewrite it as $$F(x)=W\sum_{j=1}^Ke_je_j^Tf(e_j^TAx+b_j)$$ where $\{e_i\}$ is the natural basis for $\mathbb{R}^K$ and $b=[b_i]$. This is a ...
user avatar
  • 3,100
2 votes

Confusion about convergence for the logarithm of a matrix

Gary supplied the missing link I needed; thank you, Gary! Pick up at the sequence of inequalities in my question: $$||\text{log}A||\le \sum_{m=1}^\infty ||(-1)^{m+1} \frac{(A-I)^m}{m}||=\sum_{m=1}^\...
user avatar
  • 179
2 votes

Matrix Inversion of $f(X) = (X^{-1}-A)$

You could approach this problem using differentials $$\eqalign{ \def\X{X^{-1}} F &= \X - A \\ dF &= -\X\;dX\;\X \\ }$$ Since you want the updated value of the function $F_+$ to equal zero $$dF ...
user avatar
  • 27.5k
1 vote
Accepted

Trying to understand why a certain matrix derivative is sparse

Welcome to MSE :D Simple way to deal with derivatives involving matrix multiplications is to view it via the summation form $$ Y=XW^T\\ Y_{ij}=\sum_k^{1024}X_{ik}W_{jk} $$ So what you mean by $\...
user avatar
1 vote

Reverse engineering a matrix in RREF

For example, suppose the RREF is $$ \pmatrix{1 & 0 & 1\cr 0 & 1 & 1\cr 0 & 0 & 0\cr}$$ and your matrix with first two columns known is supposed to be $$ ...
user avatar
1 vote

Derivative of diagonal matrix expression: $f(X)=\text{diag}(X)M^T\text{diag}^{-1}(MX)$

One way to simplify the matter is to use subscripts (einstein summation rule) when dealing with matrix function and derivative. For example your function could be re-writen as such $$ f(X)_{ij}=X_i M_{...
user avatar
1 vote
Accepted

What the rank of a matrix with the elements of one column to be infinity?

The following two statements are in direct contradiction: Suppose the $m\times m$ real matrix $A$ if all the elements in the first column and first row are assigned to be infinity Infinity is not a ...
user avatar
  • 114k
1 vote
Accepted

Calculate $\frac{\partial}{\partial \mathbf{A}}\lVert \mathbf{A}^{\top}\mathbf{AX}-\mathbf{X} \rVert _{F}^{2}$

Stripping away the norm, you're attempting a least-squares solution of $$A^TAX = X \quad\implies\quad A^TA = XX^+$$ QR decomposition yields $$\eqalign{ QR &= XX^+ \\ (QR)^TQR &= (XX^+)^T(XX^+) ...
user avatar
  • 27.5k
1 vote
Accepted

summation as a matrix multiplication

$ \def\L{{L}}\def\o{{\large\tt1}}\def\p{\partial} \def\LR#1{\left(#1\right)} \def\BR#1{\Bigl(#1\Bigr)} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\Diag#1{\operatorname{Diag}\LR{#1}} \def\Lapl#1{\...
user avatar
  • 27.5k
1 vote

Problem $2.9.6$ (Perko's ODE): Show $|Y(t)| \le |Y(0)|\exp\left(\int_0^t \|A(s)\|\, ds\right)$ for $Y' = AY$

$ \def\T{{\rm tr}} \def\A{\T(A)} \def\l{\lambda} \def\qiq{\quad\implies\quad} $This well known result $$\eqalign{ &\l = \log|Y| \\ &d\l = \T(Y^{-1}\,dY) \;=\; \T(Y^{-1}AY\,dt) \;=\; \A\;dt \\ &...
user avatar
  • 27.5k
1 vote
Accepted

Problem $2.9.6$ (Perko's ODE): Show $|Y(t)| \le |Y(0)|\exp\left(\int_0^t \|A(s)\|\, ds\right)$ for $Y' = AY$

For $t \ge 0$ we have $Y(t)=Y(0)+ \int_0^t A(s)Y(s)ds$, hence $$ |Y(t)| \le |Y(0)| + \int_0^t \|A(s)\|\cdot|Y(s)|ds. $$ Now (1) follows from the Grönwall-inequality: https://en.wikipedia.org/wiki/Gr%...
user avatar
  • 2,546
1 vote

Differentiating a matrix with respect to a vector

$ \def\e{\varepsilon}\def\p{\partial} \def\LR#1{\left(#1\right)} \def\BR#1{\bigl(#1\bigr)} \def\vecc#1{\operatorname{vec}\LR{#1}} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\qiq{\quad\implies\quad} \...
user avatar
  • 27.5k
1 vote

Gradient of Quadratic Function

@Jake, Regarding your question, your computations are all correct. The only thing that is missing is to recognize $$ \mathbf{x}_n^T \mathbf{w} \mathbf{x}_n = (\mathbf{x}_n \mathbf{x}_n^T) \mathbf{w} $$...
user avatar
  • 1,919
1 vote

How to take the derivative of a matrix with respect to itself?

The answer is a lot easier than the previous posters are indicating $\frac{d}{dX}(A*X)=A$ define $A=I_m$ where $m$ is the number of rows of $X$ because $I_m*X=X$ this is an identity and you get your ...
user avatar
  • 31

Only top scored, non community-wiki answers of a minimum length are eligible