Is the SVD a special case of a more general class of decompositions? $\newcommand{\on}[1]{\operatorname{#1}}$Let $V,W$ be finite-dimensional $\mathbb C$-vector spaces.
Any linear operator $A\in\on{Lin}(V,W)$ decomposes as a positive linear combination of $m\equiv \min(\dim(V),\dim(W))$ elements of the form $w v^*$, where we defined $(w v^*)(x)\equiv w \langle v,x\rangle$, with $\langle\cdot,\cdot\rangle$ the standard inner product.
More specifically, as shown via the SVD, there is always such a decomposition using at most $m$ elements.
Via the isomorphism $\on{Lin}(V,W)\simeq W\otimes V^*$, the same statement holds more generally for any tensor product of vector spaces.
In fact, for any $x\in V_1\otimes V_2$, there is a decomposition
$$x=\sum_{j=1}^{d_1}\sum_{k=1}^{d_2} c_{jk}\, (v^{(1)}_j \otimes v^{(2)}_k).$$
Let $\varphi:V_1\otimes V_2\to \on{Lin}(V_2^*,V_1)$ with $\varphi(v_1\otimes v_2) = v_1 \bar v_2^* : w\mapsto v_1 \langle\bar v_2,w\rangle$.
Then, given an arbitrary $x\in V_1\otimes V_2$, the SVD on $\varphi(x)$ should correspond to the existence of orthonormal bases $\{v^{(1)}_j\}_j\subset V_1$ and $\{v^{(2)}_j\}_j\subset V_2$ and real parameters $s_j\ge0$ such that
$$x = \sum_j s_j \,\,v_j^{(1)}\otimes v^{(2)}_j.$$
Phrased differently, this is saying that all elements of $V_1\otimes V_2$, which has dimension $d_1 d_2$, can be written as positive linear combination of $\min(d_1,d_2)$ "simple" elements of the form $v\otimes w$.
Is there a more general/abstract way to understand why this is possible at all; what this says about $V_1\otimes V_2$?
What we seem to be doing is taking a vector space $V$, and then finding a subset of simple tensors $S\subset V$ such that any $v\in V$ admits a positive decomposition in terms of elements of $S$. Is there a general theory/framework dealing with this sort of thing, of which SVD is a special case?
Actually, as pointed out in the comments, the positivity aspect isn't really insightful. So another way to phrase what I'm trying to get at is as follows: there is a specific strict subset $S\subset V\otimes W$ such that every element $x\in V\otimes W $ is a linear combination/sum of elements of $S$. More specifically, any $x$ is writable as a sum of a number of elements of $S$ equals to the rank of $x$. Moreover, here $S$ is a set of rays, albeit not a linear subspace, because $v_1\otimes v_2+v_1'\otimes v_2'\notin S$. Explicitly, $S$ is here the set of simple tensors (so to some degree, the tensor product is almost by definition the linear span of such $S$).
Is there a more general/abstract/geometric way to understand this type of structure? A more general framework in which this sort of problem is studied? I mean the more general question of a vector space $V$ being "generated" by sums of elements of some subset $S\subsetneq V$, and where one can define a notion of "cost", or rank, that provides the number of elements in the generating set required to write a given $x\in V$?
 A: Given an element $x_A\in V\otimes W$ corresponding to $A\in\text{Lin}(V, W)$, and bases $(v_i)_{i=1:n}$ and $(w_j)_{j=1:m}$, of $V$ and $W$ respectively, we can decompose
$$ x_A = \sum_{ij} \alpha_{ij}(v_i\otimes w_j) \tag{1}$$
Note that this is a sum of $m\times n$ simple tensors. We will first see that this sum can be reduced to a sum of at most $\min(m,n)$ simple tensors:

*

*keeping $i$ fixed, we can write (1) as $x_A = \sum_i \big(v_i\otimes (\sum_j \alpha_{ij} w_j)\big)$

*keeping $j$ fixed, we can write (1) as $x_A = \sum_j \big((\sum_i \alpha_{ij} v_i )\otimes w_j\big)$
Now, what does it mean for $A\in\text{Lin}(V, W)$ to be of rank $r<\min(m, n)$? It means there are some linear dependencies between the rows or columns. Say for instance, we find that the first row is a linear combination of the other rows, i.e. $\sum_{j} \alpha_{1j}w_j = \sum_{k\neq 1}\lambda_{k} \sum_{j}\alpha_{kj} w_j$. Consequently, we can get rid of a simple tensor in our sum by rewriting:
$$\begin{aligned} 
x_A &= (v_1 \otimes \sum_{j}\alpha_{1j}w_j) + \sum_{k\neq 1}(v_k \otimes \sum_{j}\alpha_{kj}w_j) 
\\&= (v_1 \otimes \sum_{k\neq 1}\lambda_k\sum_{j}\alpha_{kj}w_j) + \sum_{k\neq 1}(v_k \otimes \sum_{j}\alpha_{kj}w_j)
\\&= \sum_{k\neq 1}(\lambda_k v_1 \otimes \sum_{j}\alpha_{kj}w_j) + \sum_{k\neq 1}(v_k \otimes \sum_{j}\alpha_{kj}w_j)
\\&= \sum_{k\neq 1}(\lambda_k v_1 + v_k \otimes \sum_{j}\alpha_{kj}w_j)
\end{aligned}$$
Et voilà, now we were able to rewrite the sum using only $m-1$ simple tensors. Rinse and repeat until you remove all linear dependencies, and you'll be left with a sum of $r=\text{rank}(A)$ many simple tensors.
Showing that this is possible to do in such a way that we get orthogonality in the end probably requires some extra work.
