I hear this said in a lot of places but I can't find a proof of it anywhere. Let $V$ and $W$ be vector spaces of dimension $m$ and $n$ respectively. Then for any $t \in V \otimes W$, we have

$$ t = v_1 \otimes w_1 + \cdots v_k \otimes w_k $$

The rank of $t$ is defined to be the smallest number $r$ such that $t$ can be written as $r$ simple tensors, ie. tensors of the form $v\otimes w$.

The tensor $t$ apparently corresponds to the matrix that is the sum of the outer products. $$ v_1 w_1^T + \cdots v_kw_k^T $$ and the definition of the rank of $t$ coincides with the rank of this matrix.

Can someone show me a proof that these two notions of rank coincides or show me where I can see a proof?


This only really makes sense if $V = F^m$ and $W = F^n$ where $F = \mathbb R, \mathbb C, \mathbb Q$, or whatever is the field you do your linear algebra over, so I'm just gonna assume that $v_i, w_i$ are $m \times 1$ and $n \times 1$ column vectors.

Note that matrix multiplication is bilinear so the map $\phi\colon F^m \otimes F^n \to \mathbb M_{m,n}(F)$ that sends a simple tensor $v \otimes w$ to $vw^T$ is well defined. In fact this is an isomorphism: The standard basis of $F^m \otimes F^n$ is to take all $e_i \otimes e_j$ where $e_i$ has a $1$ in the $i^\text{th}$ row and zeros elsewhere and $e_i \otimes e_j$ gets sent to $E_{ij}$, the matrix with a $1$ in the $(i, j)^\text{th}$ position and zero's elsewhere. The $E_{ij}$ are a basis of $\mathbb M_{m,n}(F)$ so $\phi$ is an isomorphism.

Now if $t \in F^m \otimes F^n$ has rank $k$ then it can be written as $$t = v_1 \otimes w_1 + \cdots + v_k \otimes w_k$$ and the matrix $\phi(t)$ has rank at most $k$ because every column is a linear combination of the $v_i$.

On the other hand if the matrix $\phi(t)$ has rank $k$ then let $v_1, \ldots, v_k$ be a basis for it's column space. Then it's not hard to see that there exist vectors $w_1, \ldots, w_k$ such that $\phi(t) = v_1w_1^T + \cdots + v_kw_k^T$ ($w_i$ is just the coefficients of $v_i$ when the columns of $\phi(t)$ are written as linear combinations of the $v_i$). As $\phi$ is an isomorphism $t = v_1 \otimes w_1 + \cdots + v_k \otimes w_k$ which means the rank of $t$ is at most $k$.

Each definition of rank is at most the other, so they're equal.

| cite | improve this answer | |
  • $\begingroup$ Thanks this is pretty helpful. Can you just tell me why this only makes sense if $V$ and $W$ have the same dimension. Couldn't you define the map $\phi: \mathbb{F}^m \otimes \mathbb{F}^n \rightarrow \mathbb{M}_{m \times n}$ and proceed the same way. I think this map should be an isomorphism since these are both vector spaces of dimension $mn$. $\endgroup$ – jlc1112 Mar 13 '15 at 16:16
  • $\begingroup$ I think I was just confused when I wrote that, the dimensions don't need to be the same. I'll edit. $\endgroup$ – Jim Mar 13 '15 at 21:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.