How to define the set of all matrices that can be obtained by row and column swapping? Suppose that you have some (not necessarily squared) matrix $M$. I am looking for a formal way (i.e., in set builder notation) to define all the matrices $M’$ that can obtained by swapping columns and / or rows.
Let me give an example. Consider the matrix
$$M=
\begin{pmatrix}
a & b \\
c & d
\end{pmatrix}$$
Then, the set of matrices that can be obtained by row or/and column swapping are these four:

*

*By swapping only the row, we obtain
$$M_1=
\begin{pmatrix}
c & d \\
a & b
\end{pmatrix}$$


*By swapping only the column, we obtain
$$M_2=
\begin{pmatrix}
b & a \\
d & c
\end{pmatrix}$$


*By swapping both, we obtain
$$M_3=
\begin{pmatrix}
d & c \\
b & a
\end{pmatrix}$$


*By swapping neither, we obtain
$$M_4=M=
\begin{pmatrix}
a & b \\
c & d
\end{pmatrix}$$
For this particular example, the set of interest is the set of matrices $\mathcal{M}=\{M_1,M_2,M_3,M_4\}$.
I am looking for a way to construct in set builder notation the set $\mathcal{M}$ for any matrix $M$ (of arbitrary dimension).
Any help will be much appreciated.
 A: Well, I do not know to what extend you know about elementary row operations and what they really are. If you do an elementary row operation with a matrix you multiply that matrix with an invertible square matrix out of the set $\text{GL}_n(F)$ where "GL" stands for general linear group, $n$ for the size of the matrices in that set and $F$ the field the matrices in that set live in. All the matrices in that set are invertible and all matrices that are invertible (with size $n$ and over the field $F$) are in that set.
The problem is that the multiplication with these matrices give you all the elemantary row operations, i.e. adding the multiple of a row onto another row. And we do not want that. So, what you are looking for is the multiplication with the matrix $I^{i,j}=I_n-I_{i,i}-I_{j,j}+I_{i,j}+I_{j,i}$ where $I_n$ stands for the identity matrix with size $n$ and $I_{k,l}$ for the matrix (with size $n$) where alle the entries are $0$ except for the entry in row $k$ and column $l$. So, the matrix $I^{i,j}$ has is the identity matrix, but in the the entry $(i,i)$, which is on the diagonal on the $i$th row you have a $0$. In the same row however you have in the $j$th column a $1$. And on the $j$th row on the diagonal (entry $(j,j)$) again you have a $0$ and on the same row in the column $i$ you have a one. Here an example for $I^{2,4}$ for a matrix with size 5:
\begin{pmatrix}
1 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 1 & 0\\
0 & 0 & 1 & 0 & 0\\
0 & 1 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 & 1
\end{pmatrix}
If you multiply any matrix with $I^{i,j}$ it will swap the columns $i$ and $j$ in the result. So, with another example the following will give us:
$$
\begin{pmatrix}
1 & 2 & 3 & 4 & 5\\
6 & 7 & 8 & 9 & 10\\
11 & 12 & 13 & 14 & 15\\
16 & 17 & 18 & 19 & 20 
\end{pmatrix} \cdot 
\begin{pmatrix}
1 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 1 & 0\\
0 & 0 & 1 & 0 & 0\\
0 & 1 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 & 1
\end{pmatrix} =
\begin{pmatrix}
1 & 4 & 3 & 2 & 5\\
6 & 9 & 8 & 7 & 10\\
11 & 14 & 13 & 12 & 15\\
16 & 19 & 18 & 17 & 20 
\end{pmatrix}
$$
However, you would also like to have the row swaps. Well, then you have to look at the transposed matrix which basically takes your columns and turn them into rows and the other way around. It is like mirroring the matrix at the diagonal. Now you can also multiply with $I^{i,j}$ in order to change the columns of the transposed matrix. Then you have to transpose it back and have changed rows. I am not a hundred percent sure how to give a proper notation. But it might look something like this: Let $M$ be the set of all matrices that with swapped rows or columns of $A \in F^{n \times m}$. Then:
$$M=\{ A \cdot I^{i,j} | i \in {1,...,n} \text{ and } j \in {1,...,m} \text{ and } I^{i,j} \in F^{m \times m} \text{ and defined as explained above} \} \cup \{ (A^T \cdot I^{k,l})^T | k \in {1,...,m} \text{ and } l \in {1,...,n} \text{ and } I^{k,l} \in F^{n \times n} \text{ and defined as explained above} \} $$
I hope I answered your question.
EDIT:
$(A^T \cdot I^{k,l})^T=(I^{k,l})^T \cdot A =I^{k,l} \cdot A $ That is a much better notation and simpler. And also, with the set from above you only either have the row or column swaps, but for every combination and multiple of swaps you would have the following set:
$$M=\{\prod_{k=1}^n (\prod_{l=1}^n I^{k,l}) \cdot A \cdot \prod_{i=1}^m (\prod_{j=1}^m I^{i,j}) | I^{k,l} \in F^{n \times n} \text{ and } I^{i,j} \in F^{m \times m}\}  $$
Although, again I am not quite sure if I got every possible combination.
