Matrices with rows from an arbitrary vector space (and multiplication by number matrices) Matrices consist of elements of some field. However, if we have a matrix $A\in M_{m,n}(F)$, it is sometimes useful to look at each row as a vector from $F^n$, i.e., we can view the matrix as
$$A=\begin{pmatrix}\vec r_1\\\vec r_2\\\vdots\\\vec r_m\end{pmatrix}.$$ 
(Doing the same thing with columns make sense, too. I will describe stuff in this post with rows, it can be easily changed for columns.)
Sometimes it might be useful to do same thing with vectors from arbitrary vector space $V$ over a field $F$. I.e., we can use notation
$$\mathbf{B}=\begin{pmatrix}\vec v_1\\\vec v_2\\\vdots\\\vec v_m\end{pmatrix}.$$
I will use bold for "matrices consisting of vectors". 
This is just a different notation for ordered $n$-tuple of vectors. But at least in some ways they are similar to matrices.
For example, we can multiply such matrix by $A\in M_{k,m}(F)$ from the left to get
$$A\cdot \mathbf{B},$$
which is the matrix where the $i$-th row is the linear combination $\sum_{j=1}^m a_{ij}\vec v_j$. (If we choose to work with columns, we would multiply from the right.)
We can also add these matrices and multiply them by a scalar. With these definitions several properties of usual multiplication of matrices still hold - for the products that are allowed. For example, associativity $A(B\mathbf{C})=A(B\mathbf{C})$ or distributivity - both $(A+B)\mathbf{C}$ and $A(\mathbf{C}+\mathbf{D})$.
Also some properties which are valid for rank are still valid for dimension of the vector space generated by the rows. (For example, if $A$ is invertible then the "rank" of $\mathbf B$ and $A\mathbf B$ is the same. "Rank" of $A\mathbf B$ is bounded from above by the rank of $A$ and also by the "rank" of $\mathbf B$.)
We cannot multiply from the right, but we still can "cancel" on the right in the sense that if rows of $\mathbf B$ are linearly independent then $A\mathbf{B}=\mathbf{0}$ implies $A=0$ and $A_1\mathbf{B}=A_2\mathbf{B}$ implies $A_1=A_2$.
This notation can be used, for example, to make a compact notation for transition matrix between two bases by writing $\mathbf B_2=M\mathbf{B_1}$. (And some proofs about transition matrices could be written in quite a compact way using this notation. Another possible advantage of this notation is that if we are careful only to do "allowed" multiplications, than we can use many properties of the usual matrix multiplication - which after some time spend with linear algebra and matrices are used almost automatically.) 
Question. Are there some text which use this formalism, where we can multiply by "non-numerical" matrices with rows (or columns) consists of vectors from arbitrary vector space (not necessary $n$-tuples? Are there some situations when using this approach brings some advantages?
Remark 1. In a sense, the above considerations can be bypassed easily. If we work with the type of "matrices" as above, we can simply take the finite dimensional subspace $S$ which contains rows of all matrices which we need at the moment. (For example, all rows which appear in some "matrix" identity we are looking at. Or if $V$ is finitely-dimensional, we can simply take $S=V$.) If we fix some basis for $S$, this induces and isomorphism between $S$ and $F^n$, where $n=\dim(S)$ and we get a natural map $\mathbf{B} \mapsto B\in M_{m,n}$. Once we fixed a basis for $S$, any result concerning multiplication of "matrices" is now just the usual multiplication - we just need to transfer everything through this isomorphism. Still, I was curious whether sometimes it might be useful to avoid the need to fix a base and transfer things using the corresponding isomorphism.
Remark 2. Matrix summability methods can be viewed as a multiplication of an infinite matrix (of dimensions "$\mathbb N\times\mathbb N$") by a sequence considered as an infinite vector. Although in such "matrices" the rows do not have finitely many coordinates, this is different from what I have in mind, since I work here with matrices that have finitely many rows. 
 A: Take a look at the textbook Linear Algebra with Applications by John T. Scheick, from the International Series in Pure and Applied Mathematics (linked to here: https://www.amazon.com/Linear-Algebra-Applications-John-Scheick/dp/0071155309). The book is rather rare, and I think may even be out of print, but you might be able to find a second-hand copy online. Have a look at the first two chapters, and especially $\S 0.4$ on Matrix Algebra, where most of the notation is defined. The author does in fact use this style of notation quite extensively. Specifically, he denotes an $m \times n$ matrix $A$ as 
$$A \hspace{2mm} = \hspace{2mm}  
\begin{bmatrix}
  a_1 \\
  a_2 \\
  \vdots \\
  a_m \\ 
 \end{bmatrix} \hspace{2mm} = \hspace{2mm} \begin{bmatrix}
A_1 & A_2 & ... & A_n 
\end{bmatrix},$$ 
where $a_i$ is the $i$-th row, and $A_k$ the $k$-th column. There are some (minor) advantages to be gained, specifically in simple matrix-vector computations, some of which you outlined in your question. For instance, letting $B = \begin{bmatrix}
B_1 & B_2 & ... & B_n 
\end{bmatrix}$, we see that $AB = \begin{bmatrix}
AB_1 & AB_2 & ... & AB_n 
\end{bmatrix}$, allowing one to easily identify the columns of $AB.$ 
Or for another example, consider the proof that the range of a matrix equals its column space. Let $A = \begin{bmatrix}
A_1 & A_2 & ... & A_n 
\end{bmatrix},$ and let the range of $A$ be given by the usual definition of $R(A) = \left\{ y: Ax = y \text{ for some } x \text{ in the domain} \right\}.$ Then we see that the value $y$ can be expressed 
$$y = Ax = \begin{bmatrix}
A_1 & A_2 & ... & A_n 
\end{bmatrix} \begin{bmatrix}
  x_1 \\
  x_2 \\
  \vdots \\
  x_n \\ 
 \end{bmatrix} = x_1A_1 + x_2A_2 +... + x_nA_n,$$
making it crystal clear that $y$ is in the span of the columns $A_1,...,A_n$. So you save a bit of time and space from not having to write each column as an $n$-tuple, which can be handy when the computations get complicated. But aside from this, its unclear (at least to me) whether further advantages are gained from this notation. 
