What are the rules restricting the dimensions allowed for tensor multiplication? When I'm working with matrices, I know the dimensions have to match for certain operations to be defined. For matrix multiplication, if $A \in \mathbb{R}^{m\times n}, B\in \mathbb{R}^{n\times k}$, then $AB\in \mathbb{R}^{m\times k}$. For matrix addition, two matrices $C,D$ must have the same dimensions for $C + D$ to be defined.
I am confused how this works for tensors. When I say tensor, I mean a multi-dimensional array such as this one I generated in Python
[[[1 1 7]
  [9 9 3]
  [6 7 2]]

 [[0 3 5]
  [9 4 4]
  [6 4 4]]

 [[3 4 4]
  [8 4 3]
  [7 5 5]]]

I assume for tensor addition, the same rule applies. For rank $m$ tensors, $T_{1}+T_{2}$ is only defined if the dimensions are the same, i.e. $T_{1},T_{2}\in\mathbb{R}^{d_{1} \times \cdots \times d_{m}}$ for dimensions $d_{1},...,d_{m}$,
What are the rules for multiplication?
More specifically, if I have tensor
$T_{1} \in\mathbb{R}^{a_{1} \times \cdots \times a_{m}}$ and $T_{2} \in\mathbb{R}^{b_{1} \times \cdots \times b_{n}}$
what are the restrictions on $m,n$ and the $a$s and $b$s for tensor multiplication to be defined?
 A: The multiplication you can reasonably define depends on what type of tensors you have. As mentioned in the other answers for any tensors you can form the outer product of these tensors, which is just the tensor product, that is, you have two elements
$$
T \in V_1 \otimes \cdots \otimes V_n, S \in W_1 \otimes \cdots \otimes W_m 
$$
then $T \otimes S \in (V_1 \otimes \cdots \otimes V_n) \otimes (W_1 \otimes \cdots \otimes W_m)$. This generalizes the outer product of two vectors, where if $t^i \in \mathbb R^n$, $s^j \in \mathbb R^m$ then
$$
(t \otimes s)^{ij} = t^is^j.
$$
This outer product is really all you can say in the general case. Now suppose we have the situation where
$T \in V_1 \otimes V_2 \cdots \otimes V_n$ and $S \in V_1^* \otimes W_2 \otimes \cdots \otimes W_m$, so ($W_1 = V_1^*$). If we consider simple tensors
$$
v_1 \otimes \cdots \otimes v_n \in V_1 \otimes V_2 \cdots \otimes V_n,\  f \otimes w_1 \otimes \cdots \otimes w_m \in V_1^* \otimes W_2 \otimes \cdots \otimes W_m
$$
then we can define multiplication by contracting the first vector with the element of the dual space, so
$$
(v_1 \otimes \cdots \otimes v_n)(f \otimes w_1 \otimes \cdots \otimes w_m) = f(v_1) (v_2 \otimes \cdots v_n \otimes w_2 \cdots \otimes w_n) \in V_2 \otimes \cdots \otimes V_n \otimes W_2 \otimes \cdots \otimes W_m
$$
Which we can then extend to arbitrary tensors by linearity. If also $V_i$ = $W_j^*$ then you can define other multiplications (or contractions) by choosing which elements to contract. This generalizes matrix multiplication. To see this, note that for vector spaces $V, W, X$
$$
\text{Hom}(V, W) = V^* \otimes W, \ \text{Hom}(W, X) = W^* \otimes X
$$
where $\text{Hom}(V, W)$ is the set of linear maps between $V$ and $W$. Then if you multiply tensors as above from these spaces, you'll get an element in $V^* \otimes X$, which can be seen as a linear map $V \to X$. If you write all this out with simple tensors using bases and dual bases you can check that the two definitions of multiplication agree.
A: I am not familiar with the term of tensor.
Anyway, for some models (Markov Networks), I am using the same kind of objects for which there exists a multiplication operator. Maybe that will help.
This multiplication has no restriction at all. You can multiply any pair of tensors.
Actually, tensors are associated not only with dimensions $d_1,d_2, \cdots, d_n$ but more precisely with variables from a common set of variables, each variable having a cardinality that is common for all tensors associated with it.
To describe the multiplication, I need to define the notion of valuation as a set of variables associated with valid values. Two valuations $v_1$ and $v_2$ are said to be compatible if their common variables are associated with the same values. It is clear that each value within a tensor is bijectively associated with a unique valuation of its variables.
First, we define the multiplication in the case when the variables of $T_2$ are all in $T_1$. We multiply the parameters of $T_1$ by the only value of $T_2$ whose associated valuation is compatible with that of $T_1$.
In the general case, the multiplication of two tensors $T_1$ and $T_2$ is a tensor associated with the union of the variables of $T_1$ and $T_2$.
$T_1 T_2$ is initialized with 1s, then we multiply this tensor by $T_1$ then by $T_2$ to get the result.
A: The components of a tensor are numbers (real or complex). So one can multiply tensors of any rank or any variance.
There are covariants tensors whose components are indexed below as
$$T_i\quad ,\quad S_{ij}\quad ,\quad R_{ijk}.$$
There are also contravariant ones, whose indexes are above as
$$T^i\quad , \quad S^{ij}\quad ,\quad R^{ijk}.$$
But also there are tensors of mixed variance like
$$T^i{}_j\quad ,\quad S_i{}^j\quad ,\quad R^{ij}{}_k\quad ,\quad L_i{}^{jk}\quad ,\quad M^{ij}{}_{kl}.$$
So, for example, if you have $T$ as a rank 2 mixed tensor with components
$T^i{}_j$ and $S$ as a rank 3 covariant tensor with components $S_{klm}$ then their tensor product $T\otimes S$ is a rank 5 mixed tensor that has components
$$(T\otimes S)^i{}_{jklm}=T^i{}_j\cdot S_{klm},$$
where this last operation is the simple multiplication of (real or complex) numbers.
