Questions tagged [vectorization]
The vectorization of a matrix is a linear transformation that converts the matrix into a column vector.
61
questions
1
vote
0
answers
19
views
How does $\text{rank}(A)$ relate to $\text{rank}(\text{Diag}(\text{Vec}(A)))$?
Basically the title.
For $A \in \mathbb{R}^{n \times n}$, how does $\text{rank}(A)$ relate to $\text{rank}(\text{Diag}(\text{Vec}(A)))$ where $\text{Diag}(\text{Vec}(A) \in \mathbb{R}^{n^2 \times n^2}...
- 448
0
votes
1
answer
32
views
Equivalent form of vectorization of Kronecker product
This question is hard to ask, but I'll try to be as specific as I can.
For matrices $A \in \mathbb{R}^{m \times n}$, $B \in \mathbb{R}^{p \times q}$, is there an equivalent representation (see below) ...
- 448
2
votes
0
answers
55
views
Can we write down $\mathrm{vec}(Y^{-1})$ in terms of $\mathrm{vec}(Y)$?
Lets denote the Kronecker product by $\otimes$ and the vectorization of a matrix $Y$ by $\mathrm{vec}(Y)$.
Given $A\in\mathbb{R}^{n\times n}$ and $B\in\mathbb{R}^{n\times m}$, where $n\geq m$. What ...
- 1,609
1
vote
1
answer
65
views
Vector-matrix differentiation and vectorisation
In recurrent neural network backpropagation (BPTT), we have the equations:
\begin{align}
e_t &= E^T x_t \\
a_t &= W_{hx}^T e_t+ W_{hh}^T h_{t-1}\\
h_t &= \text{tanh}(a_t) \\
s_t &= W_{...
- 239
0
votes
1
answer
53
views
Decomposing a matrix vectorization
I need to compute the derivative of a matrix by a vector and I'm using matrix vectorization to do so. Define this derivative as $$ \frac{\partial vec[G(x)]}{\partial vec[(x^T)]} $$ where $x \in \...
2
votes
1
answer
43
views
Derivative of determinate function w.r.t. matrix with vectors
For my research, I need to calculate a derivative of scalar determinate function w.r.t. matrix with vectors
Here is a scalar defined as $c = \sqrt{det(A)}$
$\mathbf{A}=\mathbf{JJ}^T$
where $\mathbf{J}$...
- 23
0
votes
0
answers
20
views
Vectorizing a matrix that is not full column rank
Let $A\in\mathbb{R}^{n\times m}$ be a matrix that is not full column-rank $\text{rank}(A) = k$ for some $k < m$.
Now I vectorize this matrix
$$
a = \text{vec}(A) \in\mathbb{R}^{nm\times 1}.
$$
Is ...
- 4,396
1
vote
1
answer
107
views
Writing double summation in matrix form
I have the following weighted residuals sum of squares (WRSS)
$$ \sum_{t=1}^n \sum_{i=1}^n \left(y_i - \boldsymbol{x}_i^T \hat{\boldsymbol{\beta}_t} \right)^2 (z_t - z_i) $$
where $y_i \in \mathbb{R}^{...
- 47
0
votes
0
answers
56
views
Relationship between Kronecker square and vec square
Let $X$ be a $n$ by $n$ symmetric matrix. Let vec$(X)$ denote the operator that stacks the columns of $X$ into a row vector and let $\otimes$ denote the well-known Kronecker product.
The two $n^2$ by ...
- 554
0
votes
2
answers
140
views
Python code for Duplication matrices
I was looking for a resource/code/script in python for forming duplication matrices but surprisingly it still evades me. The wiki page gives a pseudo-code for generating this 'unique' duplication ...
- 434
3
votes
1
answer
40
views
Is there a (nice) closed form solution to $\frac{dP(t)}{dt} = AP(t)+P(t)A^T+R$
The problem: Consider constant matrices $A,R\in\mathbb{R}^{n\times n}$. I want to solve:
$$
\frac{dP(t)}{dt} = AP(t)+P(t)A^T+R
$$
for $P(t)\in\mathbb{R}^{n\times n}$.
My attempt: note that the ...
- 1,635
0
votes
0
answers
140
views
vecp vs vech operator on symmetric matrices
Let $\mathbf{X}$ $(p \times p)$ be a symmetric matrix.
The vecp operator stacks the elements of $\mathbf{X}$ above and including the diagonal columnwise.
The vech operator stacks the elements of $\...
- 554
0
votes
0
answers
151
views
Vectorizing Huber Loss function
I have the following system to minimize with respect to $\beta$ ($\phi$ and $\mathrm{y}$ are known) using Huber loss
$
\mathrm{argmin}_{\beta\in R^2}(\phi\beta-\mathrm{y})
$
where, for example,
$
\...
- 33
1
vote
2
answers
76
views
A matrix calculus problem with vectorization
I'm trying to find out the derivative of $vec(AA^T)$ w.r.t to $vec(A)$, where $A$ is a $m$ by $n$ matrix
I use a simple $3$ by $2$ case and find it entry-wisely.
I guess the answer would be $A^T\...
- 73
3
votes
1
answer
200
views
On the relation between the vectorization and the half vectorization
From Matrix Variate Distributions by Gupta & Nagar.
1) definition of vectorization for a generic matrix (page 9)
Let $X$ be a $m\times n$ matrix and let $X_1$, $\dots$, $X_n$ be the columns of $X$ ...
- 439
0
votes
1
answer
84
views
Derivative of Vectorization of matrix products w.r.t. a matrix
Suppose $\lambda \in l\times1 $， $ y \in l\times 1$，$A \in l\times mn $， $L \in m\times r $，$R \in n\times r $
$f=1/2 \parallel L\parallel_{F}^{2} + \lambda^{T} (y-A \text{vec}(LR^{T}) )$
I want to ...
- 1
0
votes
2
answers
85
views
How to "de-vectorise" a matrix? Is first column at the bottom or at the top?
I need to transform a 15x1 matrix into a 3x5 one. Is the first 3x1 column at the top, as Wikipedia seems to suggest?
https://en.wikipedia.org/wiki/Vectorization_(mathematics)
0
votes
1
answer
191
views
weighted median, but manually typed weights, not frequencies
Since theres some contoversy about the definition of the weighted median, I wonder if my doing is even possible:
I have a large 2d matrix
...
1
vote
1
answer
128
views
About standard vectorization of a matrix and its derivative
I read about this notation: if $X \in \mathbb{R}^{d\times d}$ then $X^b \in \mathbb{R}^{d^2}$ is the standard vectorization of $X$. I searched the term "standard vectorization" and only ...
- 958
1
vote
1
answer
100
views
How to do vectorization for summation for octave implementation?
I am trying to understand the transformation from a summation form to vectorization (or a form of matrix multiplications) in order to implement it in some programming language (octave or python or ...
- 135
0
votes
1
answer
84
views
Is a square commutation matrix positive semidefinite?
Let $A \in \mathbb{R}^{n \times n}$ and denote the commutation matrix, made up of 0 and 1 such that each row and each column has exactly one 1, as $K_{n} \in \mathbb{R}^{n^2 \times n^2}$ , which is ...
- 15
0
votes
1
answer
147
views
Derivative of vectorized function wrt to a Cholesky decompositiion
Let $\Sigma$ be a symmetric, positive definite $p\times p$ covariance matrix, and let $f(\Sigma)$ be it's Cholesky factor. That is, $f(\Sigma)$ is a lower triangular $p\times p$ matrix such that $\...
user802652
1
vote
1
answer
92
views
Find $\operatorname{trace}(BY^{-1})$, given $\mathrm{vec}(Y)=(A\otimes A-I_n)^{-1}\mathrm{vec}(B)$
Given diagonal $A\in\mathbb{R}^{n\times n}$, symmetric $B\in\mathbb{R}^{n\times n}$, and $\mathrm{vec}(Y)=(A\otimes A-I_n)^{-1}\mathrm{vec}(B)$, find the following:
\begin{align}
\operatorname{trace}(...
- 1,609
0
votes
2
answers
50
views
MATLAB elementwise matrix element access with ndgrid.
(Apologies if this is the wrong SE)
I've got a function containing a rather ugly nested for loop structure that I'm trying to vectorize. Effectively, I've got a matrix of coefficients, C (as a ...
- 101
1
vote
1
answer
509
views
From vec-trick to matrix-trick for Kronecker products
for the vec-trick of the Kronecker product, we can write
$$
\left(\mathbf{B}^{\top} \otimes \mathbf{A}\right) \operatorname{vec}(\mathbf{X})=\operatorname{vec}(\mathbf{A} \mathbf{X} \mathbf{B}).
$$
...
- 433
0
votes
1
answer
845
views
3D Array Multiplication
I have $2$ matrices ($A$ with shape $(5,2,3)$ and $B$ with shape $(5,3,8)$), and I want to perform some kind of multiplication in order to take a new matrix with shape $(5,2,8)$.
Pseudo Code:
...
3
votes
1
answer
189
views
Derivative of function with the Kronecker product of a Matrix with respect to vech
I have $\Sigma$ a symmetric $2 \times 2$ matrix, and $\Sigma^{-1}$ is its inverse.
Now, $\tilde{\Sigma}^{-1}=\Sigma^{-1} \otimes I_{n \times n}$ (Kronecker product).
I have a function $Y=f(\tilde{\...
2
votes
1
answer
145
views
Can I write this sum as a matrix product?
Let $\eta$ be a $n \times p$ matrix and $\Sigma$ a $p\times p$ matrix. Is it possible to rewrite the sum over element-wise quadratic forms,
$$\sum_{i=1}^n \eta_i^T \ \Sigma \ \eta_i,$$
where $\eta_i$...
- 858
0
votes
1
answer
67
views
Find $\frac{dY}{dX}, Y=(X')^{2}B$ matrix derivative
I have the following problem:
Find the matrix derivative $\frac{dY}{dX}$, where $Y=(X')^2B$, matrix $X$ is $p \times q$ and $B$ is a given matrix.
I have gotten this far:
By matrix derivative ...
- 69
1
vote
1
answer
112
views
vectorized product of two matrices, with one symmetric
For each generic non-symmetric square matrix $A$, it is well known the relation with its transpose using the commutation matrix:
$K^{(n,n)} {\rm vec}(A) = {\rm vec}(A^T)$, and of course ${\rm vec}(A) =...
- 11
0
votes
1
answer
267
views
Proof of vectorization of product of three matrices
On Wikipedia, it is written that $$\mathrm{vec}(ABC) = (C^\top\otimes A)\mathrm{vec}(B)$$
I am trying to show that it is true, but couldn't succeed. $A$ is $m\times n$, $B$ is $n\times p$ and $C$ is $...
- 14.4k
1
vote
0
answers
51
views
Kronecker products: Reordering $\text{vec}(A) \text{vec}(A)^T$ to $A \otimes A$
I'm trying to find a more elegant way to reorder the elements of the Kronecker product $\text{vec}(\textbf{A})\text{vec}(\textbf{A})^T$ into those of $\textbf{A} \otimes \textbf{A}$, where $\textbf{A}$...
- 11
1
vote
1
answer
152
views
Permuting tensor indices and relations to rearranging factors in a general-size kronecker product operators on same space?
Given the following conjecture, we can start considering larger than $2$ factor Kronecker products.
Let us say we define:
$$R_1\otimes R_2 \otimes \cdots \otimes R_N$$
And then the operation "...
- 24.3k
2
votes
2
answers
313
views
Vectorization of a diagonal matrix
Is there any representation for the vectorization of a diagonal matrix $\text{vec}(\text{Diag}(x))$?
For example, for two elements $$\text{vec}(\begin{bmatrix} x_1 & 0 \\ 0 &x_2 \end{bmatrix})...
- 523
0
votes
0
answers
43
views
Any body knows the name of the product $A\circ B$, It is not Hadamard tensor product
I have seen this operation in a vectorization operation of $D=AX^{T}B$ i.e. $vec(D)=\left( A\circ B\right) vec(X),$
\begin{equation*}
A\circ B=
\begin{pmatrix}
A_{1}B_{1}^{T} & A_{2}B_{1}^{T} &...
- 1
1
vote
2
answers
51
views
Vectorized form of a Graph's Degree matrix?
A Graph's diagonal Degree matrix, $D$ is typically defined as
$$
D_{ii} = \sum_{j} A_{ij}
$$
i.e., each i-th diagonal of the Degree matrix is the sum of the i-th row of matrix $A$ (the Adjacency ...
- 1,290
2
votes
2
answers
205
views
Derivative with respect to vectorized inverse Kronecker product
I am trying to derive the gradient of a function I wish to optimize, and wish to obtain the following derivative:
$$
\frac{\partial}{\partial \pmb{x}} \left(\pmb{I} - \pmb{X} \otimes \pmb{X} \right)^{-...
- 205
0
votes
1
answer
207
views
Derivative of vectorized block matrix in terms of derivatives of vectorized blocks
Suppose I have some block matrix $\pmb{Y}$ that is a function of $\pmb{x}$:
$$
\pmb{Y} = \begin{bmatrix}
\pmb{A} & \pmb{C} & \pmb{E} \\
\pmb{B} & \pmb{D} & \pmb{F} \\
\end{bmatrix}.
$$
...
- 205
2
votes
2
answers
483
views
Derivative of matrix w.r.t. its own vectorized version
I am unable to find what would be the derivative of a $m \times m$ real matrix $A$ with respect to $(\mathrm{vec}(A))^T$ (where $T$ is transpose and $\mathrm{vec}$ stacks the columns) without using ...
- 180
0
votes
0
answers
157
views
Vectorization identity: Weyl matrices
Take a ring $Z_n = \{0,1,...,n-1\}$. All operations occur modulo $n$. Consider the $n^{th}$ roots of unity $\omega_n = \exp\left(\frac{2\pi i}{n}\right)$. Now define
$$X = \sum_a \vert{a+1}\rangle\...
- 1,552
0
votes
0
answers
86
views
Vectorization identity proof
I'm trying to prove the identity $\vert vec(AXB)\rangle = A\otimes B^T \vert vec(X)\rangle$, where $\vert vec(L)\rangle := \sum_{ij} L_{ij}\vert i\rangle\vert j\rangle$ for any $L:= \sum_{ij}L_{ij}\...
- 1,552
-1
votes
1
answer
127
views
The location of the diagonal elements in the half-vectorization of a matrix
I'm trying to find a formula for the index of the diagonal entry when organized in a vector given by the half-vectorization operator.
Assume that we have a symmetric matrix
$$ M = \begin{bmatrix}m_{...
- 1,282
1
vote
1
answer
1k
views
Cost function - vectorized implementation
I have a problem regarding how to vectorize, more specifically the problem below:
Repeat {
$$\theta_j := \theta_j - \frac{\alpha}{m} \sum\limits_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})x_j^{(i)}$$
}
...
- 13
0
votes
1
answer
164
views
vectorization of matrices
$vec(\boldsymbol{\beta}\boldsymbol{\sigma}_{n-1}^2\boldsymbol{e}_j'\boldsymbol{\sigma}_{n-1}^2\boldsymbol{\alpha}_1^\prime)=((\boldsymbol{\alpha}_1\boldsymbol{e}_j')\otimes\boldsymbol{\beta})vec(\...
- 153
-1
votes
1
answer
42
views
How do I use vectorization to simplify matrix integration problem?
Can someone show the detailed procedures for proof:
\begin{equation*}
\text{vec}\left(\int^T_0ds\,e^{-Ks}\Sigma\Sigma^\text{T}e^{-K^\text{T}s}\right) = \left(K\otimes I+I\otimes K\right)^{-1}\text{...
- 187
5
votes
1
answer
2k
views
Vectorization and transpose: how are $\text{vec}(W^T)$ and $\text{vec}(W)$ related?
In solving for a gradient, I ended up with a differential that looks similar to:
$$
dT = (a^T \otimes b^T)\ \text{vec}[d[W]^T] + (b^T \otimes c^T)\ \text{vec}[d[W]]
$$
and I am trying to solve for $\...
- 278
0
votes
2
answers
96
views
Solve vectorial equations
I'm trying to figure out a passage for reducing a vectorial equations ! for doing this somebody told me to use a program of symbolic calculation (matlab,maple, mathematical .. or python as well .. ) I ...
1
vote
0
answers
136
views
Vectorization of expressions: How to develop a visual intution
I'm currently analyzing the expressions for backpropagation in machine learning and it takes me a lot of time to convert my derived formulas into matrices. I can't find anything helpful on google, ...
- 251
3
votes
1
answer
323
views
Finding Hessian of tr ((AB)' (AB))
I'm trying to find Hessian of $\text{tr}((AB)' (AB))$ where $A,B$ are matrices. There are nice expressions for $H_{AA}$ and $H_{BB}$ using standard approach from Magnus 1 , can anyone suggest how to ...
- 4,450
1
vote
1
answer
106
views
Derivative eigensystem
I have a diabatic potential energy matrix, $V(r,Z)$, (real symmetric) for a 2-level system with two nuclear coordinates, $(r,Z)$:
$$
V(r,Z)=
\begin{pmatrix}
V_{11}(r,Z) & V_{12}(r,Z)\\
V_{12}(r,...
- 13