Tagged Questions
0
votes
0answers
33 views
Subgradient of matrix $l1$-norm
Let $X$ be a square matrix, what is the sub-gradient of $f(X) = ||AX-XB ||_{l1}$?
$A$ and $B$ are both constant matrix.
I am very confuse about the chain rule on matrix derivatives.
0
votes
2answers
46 views
Transpose of matrix inverse: $(AA^T)^{-1}A^Tb \stackrel{?}{=} (A^TA)^{-1}A^Tb$
Given the matrix equation:
$$ x^TA^TA = b^TA $$
I'm trying to find the least squares solution (i.e.; trying to minimize $r=||Ax-b||$). The matrix $A$ is not necessarily symmetric.
When I solve it ...
0
votes
0answers
34 views
Derivative of $(Y-HX)^\top C(Y-HX)$ by $X$
I'm trying to derive an expression for
$$\nabla_X(Y-HX)^\top C(Y-HX).$$
$Y$ and $X$ are column vectors of size $N \! \times 1$.
$H$ and $C$ are matrices of size $N \! \times N$.
I have checked this ...
3
votes
2answers
55 views
Matrix Derivative of $ABC$ with respect to $B$
I have looked throughout the matrix cookbook and other sources, but am a bit confused by this problem. If I have a function $F = ABC$, what is the partial derivative of $F$ with respect to $B$? When ...
0
votes
0answers
43 views
Calculus with Exponential Matrix
I have a following with derivating and integrating using exponential Matrix. Kindly have a look at it.
Consider $A\in \mathbb{R}^{n\times n}$, $B\in \mathbb{R}^{n\times m}$, $u(t)\in ...
2
votes
1answer
70 views
Prove the mapping $(x,y,z)\mapsto (x+e^y,y+e^z,z+e^x)$ is locally invertible.
Show that the mapping $\mathbb{R}^3\to \mathbb{R}^3$, $(x,y,z)\mapsto (u,v,w)$ which is defined by $$\begin{align*}
u&=x+e^y\\
v&= y+e^z\\
w &=z+e^x
\end{align*}$$is locally invertible ...
1
vote
1answer
44 views
Derivative of a scalar function with resepct to a Matrix
I need help with the following differentiation
$$
\text{trace}((aI+bXX^T)^{-1}(aI+XX^T))
$$
with respect to $X$, where $a,b$ are some positive constants, and $I$ is the identity matrix.
Thank you
1
vote
1answer
65 views
Partial derivative with respect to a vector x for $F(x) = x^TA(x)x$
I have the next function $F(x) = x^TA(x)x$, where $x$ is a real vector with dimension $n$, and $A$ is a square real matrix $n \times n$ depending on the components of $x$.
How can I compute the ...
1
vote
1answer
93 views
Derivative of trace of matrix product including inverse
Let $A,B,X$ be n-by-n matrices, $X$ is nonsingular so $X^{-1}$ exist.
What will $\frac{\partial Tr(XAX^{-1}B)} {\partial X}$ be?
0
votes
0answers
41 views
Matrix derivatives
I have a scalar function with is defined as $$V(x,t)=x(t)^TP(t)x(t).$$ $x$ is a column vector and $P$ is a $n$-by-$n$ matrix.
I need to take $dV/dt$. I figured that $$\frac{dV}{dt} = V_x\frac{dx}{dt} ...
1
vote
2answers
109 views
Derivative with respect to a matrix
How do we start with the matrix differentiation of this kind of equation?
$$
V = [y_t-Cx_t]^TR^{-1}[y_t-Cx_t]
$$
here $x_t$ and $y_t$ are vectors and $C$ and $R$ are matrices. R is a covariance matrix ...
0
votes
0answers
19 views
Matrix $L_1$ norm derivative [duplicate]
Say that $A\in \mathcal{M}_{n,n}$ what is the result of the following derivative:
$\frac{\partial \|A- diag(A)\|_1}{\partial A}$,
where $diag(A)$ is the matrix that contains the diagonal entries of ...
0
votes
1answer
36 views
Differential and derivative of $X^{-2}$
Determine the differential and derivative of $F(X) = X^{-2}$ in which the variable X is an
n x n-matrix.
I computed the differential by using the product rule. So I first wrote
$$
f(X)= X^{-1} X^{-1}
...
1
vote
2answers
72 views
Differential and derivative of the trace of a matrix
If $X$ is a square matrix, obtain the differential and the derivative of the functions:
$f(X) = \operatorname{tr}(X)$,
$f(X) = \operatorname{tr}(X^2)$,
$f(X) = \operatorname{tr}(X^p)$ ...
4
votes
3answers
120 views
Differentiate $f(x)=x^TAx$
Calculate the differential of the function $f:\Bbb R^n\rightarrow\Bbb R$ given by $f(x)=x^TAx$, with $A$ symmetric. Also differentiate this function to $x^T$.
How exactly does this work in the case ...
0
votes
0answers
87 views
Differential and derivative of matrix function
Compute the differential and the derivative of the following function: $f: M(m \times n) \rightarrow\Bbb R$ which is given by $f(X) = \cos(b(\text{transpose})X(\text{transpose})AXb)$, where $A$ is a ...
2
votes
1answer
50 views
Derivative of $F(Ax)$
What is the identity for $$ \frac{\partial \mathbf{F}(\mathbf{A}\mathbf{x})}{\partial \mathbf{x}} = ?$$
If $\mathbf{A} \in \mathbb{R}_{mn}$, $\mathbf{x} \in \mathbb{R}_n$, and $\mathbf{F}: ...
1
vote
2answers
36 views
Derivative of a function from $M(n\times n) \to \mathbb{R}$
Given is the function $f_{i,j}: M(n\times n) \to \mathbb{R}$ defined as the $(i,j)$-th element of $X^2$. Give the differential and the derivative of $f$.
I don't have a clue where to start, since I ...
2
votes
1answer
85 views
Proving a rule about del operator as applied to matrices
How can I prove the following easily?(If it is true of course.)
\begin{align}
\nabla_{\mathbf{x}_k} \left( \sum_{i=1}^{n}\sum_{j=1}^{n} \mathbf{x}^{T}_i \mathbf{W}_{ij} \mathbf{x}_j ...
7
votes
3answers
357 views
Derivative of determinant of a matrix
Good morning everyone,
I would like to know how to calculate:
$\frac{d}{dt}\det \big(A_1(t), A_2(t), \ldots, A_n (t) \big)$
Help me please.
Thank you
1
vote
0answers
84 views
Why is the subdifferential of norm of a matrix ||A|| defined like this?
I read in a paper called "Characterization of the subdifferential of some matrix norms"
that it defines the subdifferential of the matrix norm like this:
$$\partial ||A||=\{G \in R^{m\times n} : ...
1
vote
1answer
116 views
Derivative of ratio of functions, $f,g:\mathbb{R}^N\to\mathbb{R}$ with respect to a vector?
This is fairly simple, but my matrix calculus is not that strong. Given two functions, $f:\mathbb{R}^N\to\mathbb{R}$, $g:\mathbb{R}^N\to\mathbb{R}$, and $x\in\mathbb{R}^N$, how do I compute the ...
2
votes
2answers
73 views
How to differentiate $\frac {\partial \mathrm{tr}(Q^TQAQ^TQA)}{\partial q_i}$
The problem is $\frac {\partial \mathrm{tr}(Q^TQAQ^TQA)}{\partial q_i}$, where $Q=[q_1,...,q_N]$, $q_i$ is $N$ dimensional vector and $Q$ is $N\times N$ matrix.
I have think of using chain rule, but ...
1
vote
1answer
59 views
Determinant of function composed with its inverse
Let $F:A \to B$ map subsets of $\mathbb{R}^n$ with inverse $F^{-1}$.
Let $d(\cdot) = \text{det} \mathbf{D}F(\cdot)$ with $\mathbf{D}$ denoting the total derivative matrix.
Am I correct that $d ...
2
votes
0answers
196 views
Chain rule for matrix - i'm confused
I googled around and searched inside the forum but I'm still confused about a problem.
I have 2 matrix functions $f,g : \mathbb{R}^{n \times n} \times \mathbb{R}^{a \times b} \rightarrow ...
2
votes
2answers
86 views
How to do $\frac{ \partial { \mathrm{tr}(XX^TXX^T)}}{\partial X}$
How to do the derivative
\begin{equation}
\frac{ \partial {\mathrm{tr}(XX^TXX^T)}}{\partial X}\quad ?
\end{equation}
I have no idea where to start.
2
votes
1answer
71 views
The derivative of characterestic polynomial?
Let $A\in M_{n}(R)$ and $f(x)$ be the characterestic polynomial of $A$. Is it true that $f'(x)=\sum_{i=1}^{^{n}}\sum_{j=1}^{n}\det(xI-A(i\mid j))$ which $A(i\mid j)$ is a submatrix of $A$ obtained by ...
0
votes
0answers
86 views
Differential of transposed matrices
I'm puzzling about how to deal with the differential of a transposed matrix.
I was wondering if there is some rule such that $d(X^{T}) = (dX)^{T}$.
In general I work with derivation on the trace of a ...
3
votes
2answers
186 views
Derivative of a trace
I'm new here, so "Hi" to everyone :D
I got the following problem.
I have the matrices $A$, $B$, $C$, $X$ and $Y$. All matrices are square (say n-by-n).
In particular:
- $A$ is full rank
- $B$ is ...
1
vote
0answers
67 views
Derivative of multivariate normal density wrt a scalar
Given a covariance matrix $$\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)$$ where $\sigma,\lambda \in \mathbb{R}$ (both positive), $I$ is the $n \times n$ identity matrix and $K_{\theta}^{-1}$ ...
1
vote
1answer
71 views
Finding the scalar derivative of a matrix product
I'm trying to find $$\frac{\partial}{\partial \lambda}y^T \left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y$$ where $y \in \mathbb{R^n}$ is fixed, $\lambda \in \mathbb{R}$ and ...
1
vote
1answer
142 views
Gradient of squared Frobenius norm
I'd like to find the gradient of $\frac{1}{2} ||X A^T||_F^2$ with respect to $X_{ij}$.
Going by the chain rule in the Matrix Cookbook (eqn 126), it's something like
$\partial \left[\frac{1}{2} ||X ...
0
votes
1answer
86 views
Linear transformations in $P_n$
Consider the function $d/dx$ from $P_n$ (the real vector space of degree $\leq n$ polynomials in one variable $x$) to $P_{n-1}$.
a) Prove that $d/dx$ is a linear transformation.
b) Write the ...
0
votes
1answer
52 views
Proof for $df(X)/dX$ $f(X)=\operatorname{trace}{(AX^TB+C)^{-1}D}$
Denote $f(X)=\operatorname{trace}{(AX^TB+C)^{-1}D}$ and A,B,C,D are the constant matrix, X is the $R^{m*n}$ matrix.
How to prove $df(X)/dX=-B(AX^TB+C)^{-1}D(AX^TB+C)^{-1}A$
I don't know the method ...
0
votes
1answer
103 views
Matrix Derivative
Can anyone please help me find the derivative of the ABC wrt B when:
A is say 3*3 matrix
B is 3*4 matrix
C is 4*4 matrix.
Thanks
2
votes
1answer
116 views
Vector derivative
What's the derivative of $f(w)$ with respect to the vector $w$?
$$f(w)=\mathrm{tr}(ww'A) + x^{\prime}ww'x$$
Note:
$x,w$ are vectors and $A$ is a square matrix.
${}'$ indicates transpose
Thanks.
...
1
vote
0answers
68 views
Partial derivative of matrix * constant vector wrt a vector
Is there an identity for simplifying partial derivatives of the form: $\frac{\partial A(\bf{x})\bf{b}}{\partial\bf{x}}$ ?
A is a square matrix that is a function of x, and b is a constant vector.
I ...
1
vote
1answer
658 views
Derivative of Quadratic Form
For the Quadratic Form $X^TAX; X\in\mathbb{R}^n, A\in\mathbb{R}^{n \times n}$ (which simplifies to $\Sigma_{i=0}^n\Sigma_{j=0}^nA_{ij}x_ix_j$), I tried to take the derivative wrt. X ($\Delta_X X^TAX$) ...
0
votes
1answer
224 views
Derivatives of vectors involving the expectation operator - Part I
So, I am trying to take the derivative of the following equation, because it is needed in an optimization problem. I want to make sure I am on the right track. The equation is:
$$
-3 \mathbb ...
0
votes
0answers
65 views
Transform a point to a new space. How is it working?
Let us assume that you simply have a point: $(x_1, x_2).$ You also have a transformation $H,$ that maps this single point to a new point: $(y_1, y_2).$ So $y_1 = h_1(x_1, x_2),$ and $y_2 = h_2(x_1, ...
0
votes
1answer
149 views
Trying to understand the Jacobian, part I.
So I am trying to understand the Jacobian, as it relates to the transformation of random variables. The nuts and bolts are buried in calculus however.
Now, I have been reading this paper here, and I ...
2
votes
1answer
71 views
Is this vector derivative correct?
I want to comprehend the derivative of the cost function in linear regression involving Ridge regularization, the equation is:
$$L^{\text{Ridge}}(\beta) = \sum_{i=1}^n (y_i - \phi(x_i)^T\beta)^2 + ...
0
votes
1answer
286 views
Derivative for Matrix function
I have a matrix kernel function which I am trying to find the derivative to. Function is K = c * exp[-1/2 * (P(X1 - X2))' * P(X1 -X2)] where uppercase are matrices and lower case are scalars (and ' ...
4
votes
1answer
85 views
Is this function convex when the input vector is positive?
I am wondering if $f(\mathbf{x})$ is convex on the input of a vector of $n$ positive reals $\mathbf{x}$:
$$f(\mathbf{x})=\operatorname{Tr}[(\mathbf{A}+\operatorname{diag}(\mathbf{x}))^{-1}]$$
where ...
2
votes
1answer
241 views
Derivation of the derivative of a square matrix w.r.t. a vector
So I have gotten stumped on something that seems like it (should?) be easy. I am trying to find the following derivative shown below. I have scoured the wiki link on matrix derivatives, and I think my ...
1
vote
1answer
103 views
Gradient vector function using sum and scalar
Could someone take a look on my attempt to compute the gradient for:
$$f(x) = \lambda \sum_{x = 1}^n g(x_i)$$
Where $x \in \mathbb{R^d}$, $\lambda \in \mathbb{R}$ and
$$g(x_i) = \begin{cases}
x_i - ...
1
vote
0answers
126 views
Minimizing L1 Regularization
I have given a high dimensional input $x \in \mathbb{R}^m$ where $m$ is a big number. Linear regression can be applied, but in generel it is expected, that a lot of these dimensions are actually ...
1
vote
1answer
39 views
Vector derivative with power of two in it
I want to compute the gradient of the following function with respect to $\beta$
$$L(\beta) = \sum_{i=1}^n (y_i - \phi(x_i)^T \cdot \beta)^2$$
Where $\beta$, $y_i$ and $x_i$ are vectors. The ...
5
votes
1answer
294 views
Proof for the funky trace derivative : $d (\operatorname{trace} (ABA'C))$?
Given three matrices $A$, $B$ and $C$ such that $ABA^T C$ is a square matrix, the derivative of the trace with respect to $A$ is:
$$
\nabla_A \operatorname{trace}( ABA^{T}C ) = CAB + C^T AB^T
$$
...
1
vote
2answers
163 views
Log-likelihood gradient and Hessian
Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions:
$f(x) = x^T \beta$
$p(x) = ...
