Questions tagged [vectorization]

The vectorization of a matrix is a linear transformation that converts the matrix into a column vector.

Filter by
Sorted by
Tagged with
0 votes
1 answer
51 views

Derivative of the Cholesky factor

I have a symmetric positive definite matrix $L\in\mathbb{R}^{n\times n}$ and its Cholesky factor $G\in\mathbb{R}^n$ such that $L=GG^T$. Called $\mathrm{vec}:\mathbb{R}^{n\times n}\to\mathbb{R}^{n^2}$ ...
Dadeslam's user avatar
  • 776
1 vote
1 answer
35 views

How can I generalize a vector-matrix-vector product into a vector?

I've been trying to generalize a vector-matrix-vector product that represents the $i$th element of a vector $v$, but I can't figure out how to put it into a concise form. Let $v \in \mathbb{R}^{n}$ be ...
Quyed's user avatar
  • 13
1 vote
2 answers
54 views

Does this permutation matrix have a name?

Below are examples of the matrix I have in mind: $$ P_{2} = \begin{bmatrix} 1 & & & \\ & & 1 & \\ & 1 & & \\ & & & 1 \end{...
wintergreen_plaza's user avatar
0 votes
1 answer
73 views

How to vectorize a function that accepts a matrix and returns a vector in Matlab/octave/python

Given a function, that accepts a matrix and returns a vector: $g(X):\mathbb{R}^{nxm}\rightarrow \mathbb{R}^{m}$, s.t. $g^{(i)} = t\cdot X^{(i)}$; where $X$ is an $n\times m$ matrix, $t$ is a row ...
Lis_IO's user avatar
  • 3
1 vote
1 answer
99 views

Can the transpose of cropped block-Toeplitz matrix be represented as a cropped 2D convolution?

Suppose we have the following matrices defined over the field of complex numbers ($\Bbb C$): a square input matrix $\mathbf{U}$ with dimensions $n \times n$ a symmetric convolution kernel $\mathbf{H}...
Nico Deshler's user avatar
1 vote
0 answers
71 views

$\operatorname{vec}(AB) = \operatorname{vec}(A) \otimes \operatorname{vec}(B)$

I am looking for a more elegant way to confirm the following intuition: Assume that $A$ and $B$ are two square $p\times p$ matrices. It seems there should always be some matrix $C$ such that $\...
shadow1234's user avatar
1 vote
1 answer
140 views

Solve matrix equations involving vectorization and Kronecker product

I want to find solutions for matrices $A\in \mathbb{R}^{m\times n}$ and $B\in \mathbb{R}^{n\times m}$ in the following equations: $$ \left\{ \begin{matrix} A^TM_1 = A^T(AB\odot M_2) \\ ...
Mokoghost's user avatar
0 votes
0 answers
30 views

Jacobian, vectorization and the Kroenecker product

Suppose $f(U) = U^{T} A U$ with its derivative $d_{f}$ [U] $(H) = H^{T} A U + U^{T} A H$, then its Jacobian is given by $J_f(vec U) = ((AU)^{T} \oplus I) \Pi + I \oplus U^{T} A$, (*) where $\Pi = \Pi^{...
Fred f's user avatar
  • 69
0 votes
0 answers
85 views

Partial derivative of complex matrix products

I have a constraint optimization problem formulated in a diagonal matrix form: $ P_3:~ min_{x} \quad \|A X(t) - Y(t)\|^2 \\ \text{subject to} \quad X^*(t) \cdot X(t) = \mathbb{I} $ I need to ...
user3284182's user avatar
0 votes
1 answer
37 views

How to differentiate a vector related to a matrix w.r.t. a vector by using Frobenius product notation

Here is the equation I want to solve: $$\frac{d\vec{A}}{d\vec{C}}=\frac{d\mathbf{M}(\vec{C})\vec{B}}{d\vec{C}}$$ where $\vec{A} = \mathbf{M}(\vec{C})\vec{B}$ $\vec{B}$ is a constant vector $\vec{C}$ ...
CSTim's user avatar
  • 35
1 vote
0 answers
32 views

How does $\text{rank}(A)$ relate to $\text{rank}(\text{Diag}(\text{Vec}(A)))$?

Basically the title. For $A \in \mathbb{R}^{n \times n}$, how does $\text{rank}(A)$ relate to $\text{rank}(\text{Diag}(\text{Vec}(A)))$ where $\text{Diag}(\text{Vec}(A) \in \mathbb{R}^{n^2 \times n^2}...
jacob's user avatar
  • 540
1 vote
2 answers
837 views

Equivalent form of vectorization of Kronecker product

This question is hard to ask, but I'll try to be as specific as I can. For matrices $A \in \mathbb{R}^{m \times n}$, $B \in \mathbb{R}^{p \times q}$, is there an equivalent representation (see below) ...
jacob's user avatar
  • 540
2 votes
0 answers
58 views

Can we write down $\mathrm{vec}(Y^{-1})$ in terms of $\mathrm{vec}(Y)$?

Lets denote the Kronecker product by $\otimes$ and the vectorization of a matrix $Y$ by $\mathrm{vec}(Y)$. Given $A\in\mathbb{R}^{n\times n}$ and $B\in\mathbb{R}^{n\times m}$, where $n\geq m$. What ...
Lee's user avatar
  • 1,910
1 vote
1 answer
110 views

Vector-matrix differentiation and vectorisation

In recurrent neural network backpropagation (BPTT), we have the equations: \begin{align} e_t &= E^T x_t \\ a_t &= W_{hx}^T e_t+ W_{hh}^T h_{t-1}\\ h_t &= \text{tanh}(a_t) \\ s_t &= W_{...
siegfried's user avatar
  • 238
1 vote
1 answer
111 views

Decomposing a matrix vectorization

I need to compute the derivative of a matrix by a vector and I'm using matrix vectorization to do so. Define this derivative as $$ \frac{\partial vec[G(x)]}{\partial vec[(x^T)]} $$ where $x \in \...
Darren DeBattista's user avatar
3 votes
1 answer
73 views

Derivative of determinate function w.r.t. matrix with vectors

For my research, I need to calculate a derivative of scalar determinate function w.r.t. matrix with vectors Here is a scalar defined as $c = \sqrt{det(A)}$ $\mathbf{A}=\mathbf{JJ}^T$ where $\mathbf{J}$...
CSTim's user avatar
  • 35
1 vote
1 answer
257 views

Writing double summation in matrix form

I have the following weighted residuals sum of squares (WRSS) $$ \sum_{t=1}^n \sum_{i=1}^n \left(y_i - \boldsymbol{x}_i^T \hat{\boldsymbol{\beta}_t} \right)^2 (z_t - z_i) $$ where $y_i \in \mathbb{R}^{...
MOHAMMED's user avatar
  • 169
0 votes
0 answers
96 views

Relationship between Kronecker square and vec square

Let $X$ be a $n$ by $n$ symmetric matrix. Let vec$(X)$ denote the operator that stacks the columns of $X$ into a row vector and let $\otimes$ denote the well-known Kronecker product. The two $n^2$ by ...
stollenm's user avatar
  • 603
0 votes
2 answers
365 views

Python code for Duplication matrices

I was looking for a resource/code/script in python for forming duplication matrices but surprisingly it still evades me. The wiki page gives a pseudo-code for generating this 'unique' duplication ...
Zero's user avatar
  • 464
3 votes
1 answer
43 views

Is there a (nice) closed form solution to $\frac{dP(t)}{dt} = AP(t)+P(t)A^T+R$

The problem: Consider constant matrices $A,R\in\mathbb{R}^{n\times n}$. I want to solve: $$ \frac{dP(t)}{dt} = AP(t)+P(t)A^T+R $$ for $P(t)\in\mathbb{R}^{n\times n}$. My attempt: note that the ...
FeedbackLooper's user avatar
0 votes
0 answers
600 views

vecp vs vech operator on symmetric matrices

Let $\mathbf{X}$ $(p \times p)$ be a symmetric matrix. The vecp operator stacks the elements of $\mathbf{X}$ above and including the diagonal columnwise. The vech operator stacks the elements of $\...
stollenm's user avatar
  • 603
0 votes
0 answers
354 views

Vectorizing Huber Loss function

I have the following system to minimize with respect to $\beta$ ($\phi$ and $\mathrm{y}$ are known) using Huber loss $ \mathrm{argmin}_{\beta\in R^2}(\phi\beta-\mathrm{y}) $ where, for example, $ \...
Aaron Ahn's user avatar
1 vote
2 answers
112 views

A matrix calculus problem with vectorization

I'm trying to find out the derivative of $vec(AA^T)$ w.r.t to $vec(A)$, where $A$ is a $m$ by $n$ matrix I use a simple $3$ by $2$ case and find it entry-wisely. I guess the answer would be $A^T\...
Chc's user avatar
  • 83
3 votes
1 answer
536 views

On the relation between the vectorization and the half vectorization

From Matrix Variate Distributions by Gupta & Nagar. 1) definition of vectorization for a generic matrix (page 9) Let $X$ be a $m\times n$ matrix and let $X_1$, $\dots$, $X_n$ be the columns of $X$ ...
Gost91's user avatar
  • 479
0 votes
1 answer
158 views

Derivative of Vectorization of matrix products w.r.t. a matrix

Suppose $\lambda \in l\times1 $, $ y \in l\times 1$,$A \in l\times mn $, $L \in m\times r $,$R \in n\times r $ $f=1/2 \parallel L\parallel_{F}^{2} + \lambda^{T} (y-A \text{vec}(LR^{T}) )$ I want to ...
leahxu's user avatar
  • 1
0 votes
2 answers
144 views

How to "de-vectorise" a matrix? Is first column at the bottom or at the top?

I need to transform a 15x1 matrix into a 3x5 one. Is the first 3x1 column at the top, as Wikipedia seems to suggest? https://en.wikipedia.org/wiki/Vectorization_(mathematics)
Svit's user avatar
  • 3
0 votes
1 answer
305 views

weighted median, but manually typed weights, not frequencies

Since theres some contoversy about the definition of the weighted median, I wonder if my doing is even possible: I have a large 2d matrix ...
HR_quantile's user avatar
1 vote
1 answer
189 views

About standard vectorization of a matrix and its derivative

I read about this notation: if $X \in \mathbb{R}^{d\times d}$ then $X^b \in \mathbb{R}^{d^2}$ is the standard vectorization of $X$. I searched the term "standard vectorization" and only ...
cgss's user avatar
  • 1,098
1 vote
1 answer
202 views

How to do vectorization for summation for octave implementation?

I am trying to understand the transformation from a summation form to vectorization (or a form of matrix multiplications) in order to implement it in some programming language (octave or python or ...
Noussa's user avatar
  • 135
0 votes
1 answer
124 views

Is a square commutation matrix positive semidefinite?

Let $A \in \mathbb{R}^{n \times n}$ and denote the commutation matrix, made up of 0 and 1 such that each row and each column has exactly one 1, as $K_{n} \in \mathbb{R}^{n^2 \times n^2}$ , which is ...
Hugo B's user avatar
  • 15
0 votes
1 answer
244 views

Derivative of vectorized function wrt to a Cholesky decompositiion

Let $\Sigma$ be a symmetric, positive definite $p\times p$ covariance matrix, and let $f(\Sigma)$ be it's Cholesky factor. That is, $f(\Sigma)$ is a lower triangular $p\times p$ matrix such that $\...
user avatar
1 vote
1 answer
108 views

Find $\operatorname{trace}(BY^{-1})$, given $\mathrm{vec}(Y)=(A\otimes A-I_n)^{-1}\mathrm{vec}(B)$

Given diagonal $A\in\mathbb{R}^{n\times n}$, symmetric $B\in\mathbb{R}^{n\times n}$, and $\mathrm{vec}(Y)=(A\otimes A-I_n)^{-1}\mathrm{vec}(B)$, find the following: \begin{align} \operatorname{trace}(...
Lee's user avatar
  • 1,910
0 votes
2 answers
73 views

MATLAB elementwise matrix element access with ndgrid.

(Apologies if this is the wrong SE) I've got a function containing a rather ugly nested for loop structure that I'm trying to vectorize. Effectively, I've got a matrix of coefficients, C (as a ...
user1150512's user avatar
1 vote
1 answer
933 views

From vec-trick to matrix-trick for Kronecker products

for the vec-trick of the Kronecker product, we can write $$ \left(\mathbf{B}^{\top} \otimes \mathbf{A}\right) \operatorname{vec}(\mathbf{X})=\operatorname{vec}(\mathbf{A} \mathbf{X} \mathbf{B}). $$ ...
yes's user avatar
  • 878
0 votes
1 answer
1k views

3D Array Multiplication

I have $2$ matrices ($A$ with shape $(5,2,3)$ and $B$ with shape $(5,3,8)$), and I want to perform some kind of multiplication in order to take a new matrix with shape $(5,2,8)$. Pseudo Code: ...
K.Pinitas's user avatar
3 votes
1 answer
294 views

Derivative of function with the Kronecker product of a Matrix with respect to vech

I have $\Sigma$ a symmetric $2 \times 2$ matrix, and $\Sigma^{-1}$ is its inverse. Now, $\tilde{\Sigma}^{-1}=\Sigma^{-1} \otimes I_{n \times n}$ (Kronecker product). I have a function $Y=f(\tilde{\...
Alejandra Rodriguez's user avatar
2 votes
1 answer
280 views

Can I write this sum as a matrix product?

Let $\eta$ be a $n \times p$ matrix and $\Sigma$ a $p\times p$ matrix. Is it possible to rewrite the sum over element-wise quadratic forms, $$\sum_{i=1}^n \eta_i^T \ \Sigma \ \eta_i,$$ where $\eta_i$...
tomka's user avatar
  • 928
0 votes
1 answer
93 views

Find $\frac{dY}{dX}, Y=(X')^{2}B$ matrix derivative

I have the following problem: Find the matrix derivative $\frac{dY}{dX}$, where $Y=(X')^2B$, matrix $X$ is $p \times q$ and $B$ is a given matrix. I have gotten this far: By matrix derivative ...
statistic's user avatar
1 vote
1 answer
222 views

vectorized product of two matrices, with one symmetric

For each generic non-symmetric square matrix $A$, it is well known the relation with its transpose using the commutation matrix: $K^{(n,n)} {\rm vec}(A) = {\rm vec}(A^T)$, and of course ${\rm vec}(A) =...
vivienne92's user avatar
2 votes
1 answer
780 views

Proof of vectorization of product of three matrices

On Wikipedia, it is written that $$\mathrm{vec}(ABC) = (C^\top\otimes A)\mathrm{vec}(B)$$ I am trying to show that it is true, but couldn't succeed. $A$ is $m\times n$, $B$ is $n\times p$ and $C$ is $...
Martund's user avatar
  • 14.7k
1 vote
0 answers
65 views

Kronecker products: Reordering $\text{vec}(A) \text{vec}(A)^T$ to $A \otimes A$

I'm trying to find a more elegant way to reorder the elements of the Kronecker product $\text{vec}(\textbf{A})\text{vec}(\textbf{A})^T$ into those of $\textbf{A} \otimes \textbf{A}$, where $\textbf{A}$...
Clair Barnes's user avatar
1 vote
1 answer
294 views

Permuting tensor indices and relations to rearranging factors in a general-size kronecker product operators on same space?

Given the following conjecture, we can start considering larger than $2$ factor Kronecker products. Let us say we define: $$R_1\otimes R_2 \otimes \cdots \otimes R_N$$ And then the operation "...
mathreadler's user avatar
  • 25.7k
2 votes
2 answers
528 views

Vectorization of a diagonal matrix

Is there any representation for the vectorization of a diagonal matrix $\text{vec}(\text{Diag}(x))$? For example, for two elements $$\text{vec}(\begin{bmatrix} x_1 & 0 \\ 0 &x_2 \end{bmatrix})...
Andrey Gorbunov's user avatar
0 votes
0 answers
44 views

Any body knows the name of the product $A\circ B$, It is not Hadamard tensor product

I have seen this operation in a vectorization operation of $D=AX^{T}B$ i.e. $vec(D)=\left( A\circ B\right) vec(X),$ \begin{equation*} A\circ B= \begin{pmatrix} A_{1}B_{1}^{T} & A_{2}B_{1}^{T} &...
core's user avatar
  • 1
1 vote
2 answers
123 views

Vectorized form of a Graph's Degree matrix?

A Graph's diagonal Degree matrix, $D$ is typically defined as $$ D_{ii} = \sum_{j} A_{ij} $$ i.e., each i-th diagonal of the Degree matrix is the sum of the i-th row of matrix $A$ (the Adjacency ...
24n8's user avatar
  • 1,455
2 votes
2 answers
258 views

Derivative with respect to vectorized inverse Kronecker product

I am trying to derive the gradient of a function I wish to optimize, and wish to obtain the following derivative: $$ \frac{\partial}{\partial \pmb{x}} \left(\pmb{I} - \pmb{X} \otimes \pmb{X} \right)^{-...
Sacha Epskamp's user avatar
0 votes
1 answer
319 views

Derivative of vectorized block matrix in terms of derivatives of vectorized blocks

Suppose I have some block matrix $\pmb{Y}$ that is a function of $\pmb{x}$: $$ \pmb{Y} = \begin{bmatrix} \pmb{A} & \pmb{C} & \pmb{E} \\ \pmb{B} & \pmb{D} & \pmb{F} \\ \end{bmatrix}. $$ ...
Sacha Epskamp's user avatar
2 votes
2 answers
708 views

Derivative of matrix w.r.t. its own vectorized version

I am unable to find what would be the derivative of a $m \times m$ real matrix $A$ with respect to $(\mathrm{vec}(A))^T$ (where $T$ is transpose and $\mathrm{vec}$ stacks the columns) without using ...
rsm's user avatar
  • 190
0 votes
0 answers
182 views

Vectorization identity: Weyl matrices

Take a ring $Z_n = \{0,1,...,n-1\}$. All operations occur modulo $n$. Consider the $n^{th}$ roots of unity $\omega_n = \exp\left(\frac{2\pi i}{n}\right)$. Now define $$X = \sum_a \vert{a+1}\rangle\...
user1936752's user avatar
  • 1,656
0 votes
0 answers
164 views

Vectorization identity proof

I'm trying to prove the identity $\vert vec(AXB)\rangle = A\otimes B^T \vert vec(X)\rangle$, where $\vert vec(L)\rangle := \sum_{ij} L_{ij}\vert i\rangle\vert j\rangle$ for any $L:= \sum_{ij}L_{ij}\...
user1936752's user avatar
  • 1,656