# Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

3,591 questions
Filter by
Sorted by
Tagged with
1 vote
34 views

• 1,835
15 views

### PCA proof, trace move

In the PCA proof, I see this move: What is the "trace" algebra/rule that allow this trace-move?
• 103
1 vote
98 views

### Deriving backpropagation equations - vectorization (regression)

I have a huge problem trying to derive the backpropagation equations. All the solutions I've found online are not detailed as I'd like, hence I'm here asking your help. First of all sorry for this ...
• 111
47 views
+50

• 115
40 views

### How can apply coördinate descent method on the finding least norm solution of a linear system?

Suppose we have a linear system $Ax=b$, where $A\in\Bbb{R}^{m\times n}$ and $b\in\Bbb R^m$. Given the possibility of multiple solutions, I want to find a least norm solution for this system by solving ...
138 views

### For matrix-valued functions, does $\frac d {dt} \exp(A(t)) = A'(t)\exp(A(t))$ imply $A'(t)A(t)=A(t)A'(t)$?

The converse (that $A$ and $A'$ commuting implies $(\exp(A)' = A'\exp(A)$) is easy to show from the series for $\exp$. In a class I'm TA for, one question on the students' exam was to find an example ...
• 5,482
13 views

### Construct pseudospectrum for fourth-order matrix

For a normal fourth-order matrix A with eigenvalues 1, 2, 4, 8, construct pseudospectrum $\Lambda_{\varepsilon}(A)$ and structured pseudospectrum $\Lambda_{\varepsilon}(A;I,A)$, $\varepsilon = 1/4$.
39 views

86 views

### Prove a Matrix Property

I'm stuck with the following problem. Let be $W\in \mathbb{R}^{n\times n}$ matrix, $n\geq 2$. $W$ is such that: $W_{ij}\geq 0$ for all $0\leq i,j\leq n$. $W_{ii}=0$ for all $0\leq i\leq n$. $W$ is ...
• 109
33 views

I am trying to take the derivative of a scalar function with respect to a matrix. The scalar function includes an augmented vector. The derivative I am interested in is: $$\frac{df([a|(\vec{b})^T\... • 3 -1 votes 1 answer 59 views ### \min_{\small X} \left\| X - Y\right\|_{2}^{2} + \left\| DX \right\|_{2}^{2} [closed] Given that \quad X, Y \in R^{N \times M} \quad D \in R^{N \times N} Finding it difficult to proceed differentiation on these matrix equation. The above form can be rewritten as follows Equivalent ... 0 votes 0 answers 22 views ### "Gap" between the x_i when it comes to maximizing x^TAx over simplex for blockwise matrix? We consider the local maximizer x^TAx over simplex. A is a symmetric N \times N matrix with 4 blocks. First block L \times L with elements 0 <a_{ij}<0.5 Second block K \times K ... 1 vote 1 answer 49 views ### Question on matrix over matrix derivation Consider recursive relations$$\mathbf{H}_k=\sigma(\mathbf{Z}_k),\ \mathbf{Z}_k=\mathbf{A}\mathbf{H}_{k-1}\mathbf{W}_k$$where \mathbf{Z}_k\in\mathbb{R}^{m\times n_k}, \mathbf{A}\in\mathbb{R}^{m\... • 441 1 vote 3 answers 69 views ### Is there a closed-form analytical solution to: Maximize y^T (X \beta)  s.t. (X \beta)^T (X \beta) = y^T y. Maximize y^T (X \beta)  s.t. (X \beta)^T (X \beta) = y^T y. Here y is a known vector with size n and X is a known n by m matrix. \beta is the unknown vector with size m we want to ... • 1,411 0 votes 0 answers 16 views ### A good way to measure similarity of self-similarity matrices of different sizes In the data analysis and optimization problem I am currently working on, I want to use following equality constraint (or as additional objective function): For real-valued matrices B and X where ... • 701 1 vote 1 answer 104 views ### Chain rule for matrices in index notation Consider recursive relations$$ \textbf{H}^t=\sigma(\textbf{Z}^t)  \textbf{Z}^t=\textbf{A}\textbf{H}^{t-1}\textbf{W}^t $$where \textbf{Z}^t\in\mathbb{R}^{m\times n_t}, \textbf{A}\in\mathbb{R}^... • 441 1 vote 0 answers 50 views ### Commutativity up to transposition Suppose I have two square matrices A, B for which the following property holds:$$AB= (BA)^\top$$Linear time-invariant systems have the property AB=BA, i.e., it is said that they commute. In my ... • 745 1 vote 1 answer 43 views ### Matrix derivation involving element-wise function Let$$ \mathbf{H}=\sigma(\mathbf{Z})  where $\sigma$ is an element-wise non-linear function, $\mathbf{H}\in\mathbb{R}^{m\times n}$, and $\mathbf{Z}\in\mathbb{R}^{m\times n}$. What is the index ...
• 441
1 vote
I want to show that the following matrix satisfies $\|A^n(z)\| \leq C$ for every $z \in \mathbb{C}$ with $Re(z) \leq 0$, for $n= 0,1,2,...$, and some constant $C$. How do I approach this kind of ...
How do I calculate this, $\beta$ is a parameter, $H$ a matrix: $(1-\beta \partial_\beta)\ln(Tr(e^{-\beta H})) = \ln(Tr(e^{-\beta H})) - \beta \frac{\partial_\beta Tr(e^{-\beta H})}{Tr(e^{-\beta H})}=?$...