Questions tagged [vectorization]

The vectorization of a matrix is a linear transformation that converts the matrix into a column vector.

Filter by
Sorted by
Tagged with
1 vote
0 answers
19 views

How does $\text{rank}(A)$ relate to $\text{rank}(\text{Diag}(\text{Vec}(A)))$?

Basically the title. For $A \in \mathbb{R}^{n \times n}$, how does $\text{rank}(A)$ relate to $\text{rank}(\text{Diag}(\text{Vec}(A)))$ where $\text{Diag}(\text{Vec}(A) \in \mathbb{R}^{n^2 \times n^2}...
user avatar
  • 448
0 votes
1 answer
32 views

Equivalent form of vectorization of Kronecker product

This question is hard to ask, but I'll try to be as specific as I can. For matrices $A \in \mathbb{R}^{m \times n}$, $B \in \mathbb{R}^{p \times q}$, is there an equivalent representation (see below) ...
user avatar
  • 448
2 votes
0 answers
55 views

Can we write down $\mathrm{vec}(Y^{-1})$ in terms of $\mathrm{vec}(Y)$?

Lets denote the Kronecker product by $\otimes$ and the vectorization of a matrix $Y$ by $\mathrm{vec}(Y)$. Given $A\in\mathbb{R}^{n\times n}$ and $B\in\mathbb{R}^{n\times m}$, where $n\geq m$. What ...
user avatar
  • 1,609
1 vote
1 answer
65 views

Vector-matrix differentiation and vectorisation

In recurrent neural network backpropagation (BPTT), we have the equations: \begin{align} e_t &= E^T x_t \\ a_t &= W_{hx}^T e_t+ W_{hh}^T h_{t-1}\\ h_t &= \text{tanh}(a_t) \\ s_t &= W_{...
user avatar
  • 239
0 votes
1 answer
53 views

Decomposing a matrix vectorization

I need to compute the derivative of a matrix by a vector and I'm using matrix vectorization to do so. Define this derivative as $$ \frac{\partial vec[G(x)]}{\partial vec[(x^T)]} $$ where $x \in \...
user avatar
2 votes
1 answer
43 views

Derivative of determinate function w.r.t. matrix with vectors

For my research, I need to calculate a derivative of scalar determinate function w.r.t. matrix with vectors Here is a scalar defined as $c = \sqrt{det(A)}$ $\mathbf{A}=\mathbf{JJ}^T$ where $\mathbf{J}$...
user avatar
  • 23
0 votes
0 answers
20 views

Vectorizing a matrix that is not full column rank

Let $A\in\mathbb{R}^{n\times m}$ be a matrix that is not full column-rank $\text{rank}(A) = k$ for some $k < m$. Now I vectorize this matrix $$ a = \text{vec}(A) \in\mathbb{R}^{nm\times 1}. $$ Is ...
user avatar
  • 4,396
1 vote
1 answer
107 views

Writing double summation in matrix form

I have the following weighted residuals sum of squares (WRSS) $$ \sum_{t=1}^n \sum_{i=1}^n \left(y_i - \boldsymbol{x}_i^T \hat{\boldsymbol{\beta}_t} \right)^2 (z_t - z_i) $$ where $y_i \in \mathbb{R}^{...
user avatar
0 votes
0 answers
56 views

Relationship between Kronecker square and vec square

Let $X$ be a $n$ by $n$ symmetric matrix. Let vec$(X)$ denote the operator that stacks the columns of $X$ into a row vector and let $\otimes$ denote the well-known Kronecker product. The two $n^2$ by ...
user avatar
  • 554
0 votes
2 answers
140 views

Python code for Duplication matrices

I was looking for a resource/code/script in python for forming duplication matrices but surprisingly it still evades me. The wiki page gives a pseudo-code for generating this 'unique' duplication ...
user avatar
  • 434
3 votes
1 answer
40 views

Is there a (nice) closed form solution to $\frac{dP(t)}{dt} = AP(t)+P(t)A^T+R$

The problem: Consider constant matrices $A,R\in\mathbb{R}^{n\times n}$. I want to solve: $$ \frac{dP(t)}{dt} = AP(t)+P(t)A^T+R $$ for $P(t)\in\mathbb{R}^{n\times n}$. My attempt: note that the ...
user avatar
0 votes
0 answers
140 views

vecp vs vech operator on symmetric matrices

Let $\mathbf{X}$ $(p \times p)$ be a symmetric matrix. The vecp operator stacks the elements of $\mathbf{X}$ above and including the diagonal columnwise. The vech operator stacks the elements of $\...
user avatar
  • 554
0 votes
0 answers
151 views

Vectorizing Huber Loss function

I have the following system to minimize with respect to $\beta$ ($\phi$ and $\mathrm{y}$ are known) using Huber loss $ \mathrm{argmin}_{\beta\in R^2}(\phi\beta-\mathrm{y}) $ where, for example, $ \...
user avatar
1 vote
2 answers
76 views

A matrix calculus problem with vectorization

I'm trying to find out the derivative of $vec(AA^T)$ w.r.t to $vec(A)$, where $A$ is a $m$ by $n$ matrix I use a simple $3$ by $2$ case and find it entry-wisely. I guess the answer would be $A^T\...
user avatar
  • 73
3 votes
1 answer
200 views

On the relation between the vectorization and the half vectorization

From Matrix Variate Distributions by Gupta & Nagar. 1) definition of vectorization for a generic matrix (page 9) Let $X$ be a $m\times n$ matrix and let $X_1$, $\dots$, $X_n$ be the columns of $X$ ...
user avatar
  • 439
0 votes
1 answer
84 views

Derivative of Vectorization of matrix products w.r.t. a matrix

Suppose $\lambda \in l\times1 $, $ y \in l\times 1$,$A \in l\times mn $, $L \in m\times r $,$R \in n\times r $ $f=1/2 \parallel L\parallel_{F}^{2} + \lambda^{T} (y-A \text{vec}(LR^{T}) )$ I want to ...
user avatar
  • 1
0 votes
2 answers
85 views

How to "de-vectorise" a matrix? Is first column at the bottom or at the top?

I need to transform a 15x1 matrix into a 3x5 one. Is the first 3x1 column at the top, as Wikipedia seems to suggest? https://en.wikipedia.org/wiki/Vectorization_(mathematics)
user avatar
0 votes
1 answer
191 views

weighted median, but manually typed weights, not frequencies

Since theres some contoversy about the definition of the weighted median, I wonder if my doing is even possible: I have a large 2d matrix ...
user avatar
1 vote
1 answer
128 views

About standard vectorization of a matrix and its derivative

I read about this notation: if $X \in \mathbb{R}^{d\times d}$ then $X^b \in \mathbb{R}^{d^2}$ is the standard vectorization of $X$. I searched the term "standard vectorization" and only ...
user avatar
  • 958
1 vote
1 answer
100 views

How to do vectorization for summation for octave implementation?

I am trying to understand the transformation from a summation form to vectorization (or a form of matrix multiplications) in order to implement it in some programming language (octave or python or ...
user avatar
  • 135
0 votes
1 answer
84 views

Is a square commutation matrix positive semidefinite?

Let $A \in \mathbb{R}^{n \times n}$ and denote the commutation matrix, made up of 0 and 1 such that each row and each column has exactly one 1, as $K_{n} \in \mathbb{R}^{n^2 \times n^2}$ , which is ...
user avatar
  • 15
0 votes
1 answer
147 views

Derivative of vectorized function wrt to a Cholesky decompositiion

Let $\Sigma$ be a symmetric, positive definite $p\times p$ covariance matrix, and let $f(\Sigma)$ be it's Cholesky factor. That is, $f(\Sigma)$ is a lower triangular $p\times p$ matrix such that $\...
user avatar
1 vote
1 answer
92 views

Find $\operatorname{trace}(BY^{-1})$, given $\mathrm{vec}(Y)=(A\otimes A-I_n)^{-1}\mathrm{vec}(B)$

Given diagonal $A\in\mathbb{R}^{n\times n}$, symmetric $B\in\mathbb{R}^{n\times n}$, and $\mathrm{vec}(Y)=(A\otimes A-I_n)^{-1}\mathrm{vec}(B)$, find the following: \begin{align} \operatorname{trace}(...
user avatar
  • 1,609
0 votes
2 answers
50 views

MATLAB elementwise matrix element access with ndgrid.

(Apologies if this is the wrong SE) I've got a function containing a rather ugly nested for loop structure that I'm trying to vectorize. Effectively, I've got a matrix of coefficients, C (as a ...
user avatar
1 vote
1 answer
509 views

From vec-trick to matrix-trick for Kronecker products

for the vec-trick of the Kronecker product, we can write $$ \left(\mathbf{B}^{\top} \otimes \mathbf{A}\right) \operatorname{vec}(\mathbf{X})=\operatorname{vec}(\mathbf{A} \mathbf{X} \mathbf{B}). $$ ...
user avatar
  • 433
0 votes
1 answer
845 views

3D Array Multiplication

I have $2$ matrices ($A$ with shape $(5,2,3)$ and $B$ with shape $(5,3,8)$), and I want to perform some kind of multiplication in order to take a new matrix with shape $(5,2,8)$. Pseudo Code: ...
user avatar
3 votes
1 answer
189 views

Derivative of function with the Kronecker product of a Matrix with respect to vech

I have $\Sigma$ a symmetric $2 \times 2$ matrix, and $\Sigma^{-1}$ is its inverse. Now, $\tilde{\Sigma}^{-1}=\Sigma^{-1} \otimes I_{n \times n}$ (Kronecker product). I have a function $Y=f(\tilde{\...
user avatar
2 votes
1 answer
145 views

Can I write this sum as a matrix product?

Let $\eta$ be a $n \times p$ matrix and $\Sigma$ a $p\times p$ matrix. Is it possible to rewrite the sum over element-wise quadratic forms, $$\sum_{i=1}^n \eta_i^T \ \Sigma \ \eta_i,$$ where $\eta_i$...
user avatar
  • 858
0 votes
1 answer
67 views

Find $\frac{dY}{dX}, Y=(X')^{2}B$ matrix derivative

I have the following problem: Find the matrix derivative $\frac{dY}{dX}$, where $Y=(X')^2B$, matrix $X$ is $p \times q$ and $B$ is a given matrix. I have gotten this far: By matrix derivative ...
user avatar
1 vote
1 answer
112 views

vectorized product of two matrices, with one symmetric

For each generic non-symmetric square matrix $A$, it is well known the relation with its transpose using the commutation matrix: $K^{(n,n)} {\rm vec}(A) = {\rm vec}(A^T)$, and of course ${\rm vec}(A) =...
user avatar
0 votes
1 answer
267 views

Proof of vectorization of product of three matrices

On Wikipedia, it is written that $$\mathrm{vec}(ABC) = (C^\top\otimes A)\mathrm{vec}(B)$$ I am trying to show that it is true, but couldn't succeed. $A$ is $m\times n$, $B$ is $n\times p$ and $C$ is $...
user avatar
  • 14.4k
1 vote
0 answers
51 views

Kronecker products: Reordering $\text{vec}(A) \text{vec}(A)^T$ to $A \otimes A$

I'm trying to find a more elegant way to reorder the elements of the Kronecker product $\text{vec}(\textbf{A})\text{vec}(\textbf{A})^T$ into those of $\textbf{A} \otimes \textbf{A}$, where $\textbf{A}$...
user avatar
1 vote
1 answer
152 views

Permuting tensor indices and relations to rearranging factors in a general-size kronecker product operators on same space?

Given the following conjecture, we can start considering larger than $2$ factor Kronecker products. Let us say we define: $$R_1\otimes R_2 \otimes \cdots \otimes R_N$$ And then the operation "...
user avatar
  • 24.3k
2 votes
2 answers
313 views

Vectorization of a diagonal matrix

Is there any representation for the vectorization of a diagonal matrix $\text{vec}(\text{Diag}(x))$? For example, for two elements $$\text{vec}(\begin{bmatrix} x_1 & 0 \\ 0 &x_2 \end{bmatrix})...
user avatar
0 votes
0 answers
43 views

Any body knows the name of the product $A\circ B$, It is not Hadamard tensor product

I have seen this operation in a vectorization operation of $D=AX^{T}B$ i.e. $vec(D)=\left( A\circ B\right) vec(X),$ \begin{equation*} A\circ B= \begin{pmatrix} A_{1}B_{1}^{T} & A_{2}B_{1}^{T} &...
user avatar
  • 1
1 vote
2 answers
51 views

Vectorized form of a Graph's Degree matrix?

A Graph's diagonal Degree matrix, $D$ is typically defined as $$ D_{ii} = \sum_{j} A_{ij} $$ i.e., each i-th diagonal of the Degree matrix is the sum of the i-th row of matrix $A$ (the Adjacency ...
user avatar
  • 1,290
2 votes
2 answers
205 views

Derivative with respect to vectorized inverse Kronecker product

I am trying to derive the gradient of a function I wish to optimize, and wish to obtain the following derivative: $$ \frac{\partial}{\partial \pmb{x}} \left(\pmb{I} - \pmb{X} \otimes \pmb{X} \right)^{-...
user avatar
0 votes
1 answer
207 views

Derivative of vectorized block matrix in terms of derivatives of vectorized blocks

Suppose I have some block matrix $\pmb{Y}$ that is a function of $\pmb{x}$: $$ \pmb{Y} = \begin{bmatrix} \pmb{A} & \pmb{C} & \pmb{E} \\ \pmb{B} & \pmb{D} & \pmb{F} \\ \end{bmatrix}. $$ ...
user avatar
2 votes
2 answers
483 views

Derivative of matrix w.r.t. its own vectorized version

I am unable to find what would be the derivative of a $m \times m$ real matrix $A$ with respect to $(\mathrm{vec}(A))^T$ (where $T$ is transpose and $\mathrm{vec}$ stacks the columns) without using ...
user avatar
  • 180
0 votes
0 answers
157 views

Vectorization identity: Weyl matrices

Take a ring $Z_n = \{0,1,...,n-1\}$. All operations occur modulo $n$. Consider the $n^{th}$ roots of unity $\omega_n = \exp\left(\frac{2\pi i}{n}\right)$. Now define $$X = \sum_a \vert{a+1}\rangle\...
user avatar
  • 1,552
0 votes
0 answers
86 views

Vectorization identity proof

I'm trying to prove the identity $\vert vec(AXB)\rangle = A\otimes B^T \vert vec(X)\rangle$, where $\vert vec(L)\rangle := \sum_{ij} L_{ij}\vert i\rangle\vert j\rangle$ for any $L:= \sum_{ij}L_{ij}\...
user avatar
  • 1,552
-1 votes
1 answer
127 views

The location of the diagonal elements in the half-vectorization of a matrix

I'm trying to find a formula for the index of the diagonal entry when organized in a vector given by the half-vectorization operator. Assume that we have a symmetric matrix $$ M = \begin{bmatrix}m_{...
user avatar
  • 1,282
1 vote
1 answer
1k views

Cost function - vectorized implementation

I have a problem regarding how to vectorize, more specifically the problem below: Repeat { $$\theta_j := \theta_j - \frac{\alpha}{m} \sum\limits_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})x_j^{(i)}$$ } ...
user avatar
  • 13
0 votes
1 answer
164 views

vectorization of matrices

$vec(\boldsymbol{\beta}\boldsymbol{\sigma}_{n-1}^2\boldsymbol{e}_j'\boldsymbol{\sigma}_{n-1}^2\boldsymbol{\alpha}_1^\prime)=((\boldsymbol{\alpha}_1\boldsymbol{e}_j')\otimes\boldsymbol{\beta})vec(\...
user avatar
  • 153
-1 votes
1 answer
42 views

How do I use vectorization to simplify matrix integration problem?

Can someone show the detailed procedures for proof: \begin{equation*} \text{vec}\left(\int^T_0ds\,e^{-Ks}\Sigma\Sigma^\text{T}e^{-K^\text{T}s}\right) = \left(K\otimes I+I\otimes K\right)^{-1}\text{...
user avatar
5 votes
1 answer
2k views

Vectorization and transpose: how are $\text{vec}(W^T)$ and $\text{vec}(W)$ related?

In solving for a gradient, I ended up with a differential that looks similar to: $$ dT = (a^T \otimes b^T)\ \text{vec}[d[W]^T] + (b^T \otimes c^T)\ \text{vec}[d[W]] $$ and I am trying to solve for $\...
user avatar
  • 278
0 votes
2 answers
96 views

Solve vectorial equations

I'm trying to figure out a passage for reducing a vectorial equations ! for doing this somebody told me to use a program of symbolic calculation (matlab,maple, mathematical .. or python as well .. ) I ...
user avatar
1 vote
0 answers
136 views

Vectorization of expressions: How to develop a visual intution

I'm currently analyzing the expressions for backpropagation in machine learning and it takes me a lot of time to convert my derived formulas into matrices. I can't find anything helpful on google, ...
user avatar
  • 251
3 votes
1 answer
323 views

Finding Hessian of tr ((AB)' (AB))

I'm trying to find Hessian of $\text{tr}((AB)' (AB))$ where $A,B$ are matrices. There are nice expressions for $H_{AA}$ and $H_{BB}$ using standard approach from Magnus 1 , can anyone suggest how to ...
user avatar
1 vote
1 answer
106 views

Derivative eigensystem

I have a diabatic potential energy matrix, $V(r,Z)$, (real symmetric) for a 2-level system with two nuclear coordinates, $(r,Z)$: $$ V(r,Z)= \begin{pmatrix} V_{11}(r,Z) & V_{12}(r,Z)\\ V_{12}(r,...
user avatar
  • 13