How many square matrices are there with given columns, rows and diagonals Suppose we have $n\times n$ matrix that contains numbers from $1$ to $n^2$.
How many matrices are there that their $n$ columns, $n$ rows and $2n$ diagonals contain given numbers?
For example $3\times 3$ matrix:
\begin{pmatrix} 
1 & 2 & 3 \\
4 & 5 & 6 \\
7 & 8 & 9 \\
\end{pmatrix}
Its rows are $(1,2,3),(4,5,6),(7,8,9)$.
Its columns are $(1,4,7),(2,5,8),(3,6,9)$.
Its diagonals are $(1,5,9),(2,6,7),(3,4,8),(1,6,8),(2,4,9),(3,5,7)$.
If we join them all together sorted inside and sorted overall we get:
$$\big((1,2,3),(1,4,7),(1,5,9),(1,6,8),(2,4,9),(2,5,8),(2,6,7),$$$$(3,4,8),(3,5,7),(3,6,9),(4,5,6),(7,8,9)\big)$$
Question is how many $3\times 3$ matrices there are with the same sorted set?
I brute-forced it for $n=3$. And the number is $432$. But I am not able to figure out a formula for it. The number $432$ is divisible by $9$ which is of course because the original matrix can be shifted to $3\cdot 3=9$ different positions preserving the ordered set. And also by $16$ as we can rotate whole matrix $4$ times by $90$ degrees and there are $4$ mirror images for each one. $432/9/16=3$ so there must be $3$ fundamental matrices from which we can generate all $432$ by shifting, rotating and mirroring. (If I am not mistaken.)
Just one example out of $432$ that has the same sorted set as original matrix:
\begin{pmatrix}
 4 & 2 & 9 \\
 1 & 8 & 6 \\
 7 & 5 & 3 \\
\end{pmatrix}
...and perhaps additional question: How to produce all the required matrices algorithmically one by one?
 A: I have answered your question both in its literal interpretation, which has a trivial answer, and I've tried to interpret your actual question on the basis of your example, and your finding of $432$ matrices for $n=3$. For the latter, continue reading from the second horizontal line onward.

Literal interpretation, trivial answer:
Taking your question literally, the answer is $1$ for all $n$. After all, if the ordered sets 
$$(\text{rows},\ \text{columns},\ \text{diagonals},\ \text{antidiagonals}),$$
of two matrices are the same, then in particular their sets of rows and columns are the same. Of course the $i,j$-th entry of a matrix is in the $j$-th place in some row and the $i$-th place in some column, so its sets of rows and columns together determine the matrix. If we consider the unordered sets
$$\{\text{rows},\ \text{columns},\ \text{diagonals},\ \text{antidiagonals}\},$$
then there are more options; transposing a matrix swaps the sets of rows and columns, but preserves the sets of diagonals and antidiagonals. Similarly, reflecting a matrix along a horizontal (or vertical) axis preserves the sets of rows and columns, but preserves the sets of diagonals and antidiagonals. In this way we already find $4$ matrices with the same unordered sets.
Proposition: For every square matrix are either $4$ or $8$ matrices with the same unordered set.
Proof. Given two matrices $M$ and $N$ with the same unordered set, the first row $M$ is either a row, column, diagonal or antidiagonal of $N$. The other rows of $M$ are precisely the $n$-tuples in the unordered set of $M$ that do not share a coordinate with the first row. Hence they correspond to the same type (row, column, diagonal or antidiagonal) as the first row. So the matrix $N$ induces a permutation of the set
$$\{\{\text{rows of }M\},\{\text{columns of }M\},\{\text{diagonals of }M\},\{\text{antidiagonals of }M\}\}.\tag{1}$$
As noted before, a matrix is determined by its sets of rows and columns. So given $M$, the matrix $N$ is determined by whether the first row and first column of $M$ are rows, columns, diagonals or antidiagonals. This shows that conversely $N$ is uniquely determined by the permutation of $(1)$ it induces, and moreover that $N$ is determined by where it maps the rows and columns of $N$. This shows that the set of matrices with the same unordered set are in bijection with a subgroup of $S_4$ of order at most $4\times3=12$. As noted before, transposing and reflecting $M$ preserves the set $(1)$. This show that this subgroup contains a subgroup isomorphic to $\langle(1\ 2),(3\ 4)\rangle$, and hence it is isomorphic either $V_4$ or $D_8$. Hence there are either $4$ or $8$ matrices with the same unordered set.
$\hspace{10pt}\square$
The proposition could be improved by determining whether there are $4$ or $8$ such matrices, and what this depends on, if anything. But given that you found $432$ matrices with the same 'ordered set', you must mean something else. My best guess is that you take the rows, columns, diagonals and antidiagonals as unordered sets as well.

Different interpretation, interesting answer:
Given a matrix $M$, let $X$ its unordered set of unordered sets. For you example $3\times3$-matrix, this is
$$X=\left\{\begin{array}{lll}\{1,2,3\},&\{4,5,6\},&\{7,8,9\},\\
\{1,4,7\},&\{2,5,8\},&\{3,6,9\},\\
\{1,5,9\},&\{2,6,7\},&\{3,4,8\},\\
\{3,5,7\},&\{2,4,9\},&\{1,6,8\}
\end{array}\right\}.$$
In general, if $M$ is an $n\times n$-matrix then $|X|=4n$ (if $n\geq3$)${}^1$, and each element of $X$ is an $n$-element subset of $\{1,\ldots,n^2\}$. Moreover, the relation on the set $X$ consisting of the pairs
$$(x,y)\in X\times X\qquad\text{ with }\qquad x\cap y=\varnothing,$$
is an equivalence relation that partitions $X$ into $4$ subsets of $n$ elements each (if is odd)${}^2$, say $X_1$, $X_2$, $X_3$ and $X_4$. These correspond to the sets of rows, columns, diagonals and antidiagonals of $M$.
As noted above (in the first part of my answer to the literal interpretation), any ordering of the rows and columns determines the matrix entirely. So any choice of $X_i$ and $X_j$ for the rows and columns, and any ordering of $X_i$ an $X_j$ determines a matrix with the same unordered set as $M$. This yields a total of $4\times3\times n!\times n!=12(n!)^2$ matrices with the same unordered set as $M$.
Of course, different choices of $X_i$ and $X_j$ yield different matrices, as their rows and columns will differ. Similarly, different orderings of $X_i$ and $X_j$ yield different matrices, as the orders of their rows and/or columns will differ. This shows that there are precisely $12(n!)^2$ matrices with the same unordered set, and this characterisation gives you a way to construct them all explicitly.
Note that in particular, for $n=3$ this gives $12\times(3!)^2=432$ matrices, in agreement with your finding.

More explicit description of algorithm to generate all solutions:
To generate all matrices with the same unordered set $X$ as some given $n\times n$-matrix $M$, where $n\geq3$ is odd, 


*

*Define $X_1$, $X_2$, $X_3$ and $X_4$ as the sets of rows, columns, diagonals and antidiagonals.

*Choose distinct sets $X_a$ and $X_b$ for the rows and columns.

*Choose orderings of $X_a$ and $X_b$, and denote them by
$$X_a=(X_a[1],\ldots,X_a[n])
 \qquad\text{ and }\qquad
 X_b=(X_b[1],\ldots,X_b[n]).$$

*For each integer $k\in\{1,\ldots,n^2\}$, determine $i$ and $j$ such that $k\in X_a[i]$ and $k\in X_b[j]$.

*Define $n_{ij}:=k$. 

*Define $N:=(n_{ij})_{1\leq i,j\leq n}$.


The matrix $N$ now has the same unordered set $X$ as $M$, and going through every choice of $X_a$ and $X_b$ and every ordering of $X_a$ and $X_b$ yields all matrices with the same unordered set $X$.
Below is my first attempt at writing some sort of pseudo-code. In stead of making the $X_i$ sets and choosing orderings, I make the $X_i$ lists and choose permutations of them.  
Input matrix M
Define X_1 = list of rows of M        
Define X_2 = list of columns of M
Define X_3 = list of diagonals of M
Define X_4 = list of antidiagonals of M
For each {integer a in {1,2,3,4}}{               // X_a is the set of rows
   For each {integer b in {1,2,3,4} with b!=a}{  // X_b is the set of columns
      For each {permutation s in S_n}{           // s is an ordering of X_a
         For each {permutation t in S_n}{        // t is an ordering of X_b 
            For each {integer k in {1,...,n^2}}{
               Define i such that k in X_a[s(i)]
               Define j such that k in X_b[t(j)]
               Define n_ij = k
            }
            Output matrix N_{a,b,s,t} = (n_ij).
         }
      }
   }
}

This outputs all matrices with the the same unordered set $X$ as $M$.

Footnotes:
$1$: If $n\leq2$ then the sets of rows, columns, diagonals and antidiagonals are not all distinct; for $n=1$ all four sets coincide and so $|X|=1$, for $n=2$ the sets of diagonals and antidiagonals coincide and so $|X|=6$.
$2$: If $n$ is odd, then every diagonal and antidiagonal have precisely one entry in common. If $n$ is even this is not the case, and the relation is not an equivalence relation.
A: I am adding a new answer because this takes a drastically different viewpoint, which gives a very simple solution, but works only for the case of $n=3$.
After mulling over the insightful answer from @Servaes, I realized the $3\times 3$ case is a finite affine plane: https://en.wikipedia.org/wiki/Affine_plane_(incidence_geometry)#Finite_affine_planes
In particular this means any 2 points (2 cells in the matrix) defines a line (row / column / main diagonal / anti-main diagonal).  Obviously this is true only for $n=3$.
Based on this observation the counting becomes super simple:


*

*There are 9 places for 1.

*After that, there are 8 places for 2.

*After 1 & 2, they together determine 3's location.  (1 & 2 define a line, and 3 must be the remaining place in that line.)

*There are 6 places left for 4.

*After 1, 2, 3 & 4, all other numbers are determined: 1 & 4 determine 7's location, 2 & 4 determine 9's location, 3 & 4 determine 8's location, 3 & 9 determine 6's location, and 2 & 8 determine 5's location.
So the total number of such matrices $= 9 \times 8 \times 6 = 432$.
This proof is not quite complete because, technically speaking, I have to show that the assignments above do not create a contradiction.  E.g. 2 & 8 determine 5's location, but 1 & 9 also determine 5's location, and I have not shown that those two location-assignments agree.  However, if you can see why the matrix is analogous to the finite affine plane, then the symmetry between all lines would argue that the assignments must agree.  Or to use the words of lazy professors everywhere, "It's obvious." :)  (Sorry!)
A: Too long & messy for a comment...
As usual we will call the diagonal from top-left to bottom-right as the main diagonal.
One operation you didn't include (which works for $n=3$ only, I think) is swap any 2 columns (or rows).  This would maintain the row-sets and column-sets, while the main-diagonal-sets (main diagonal and parallel lines) would become the anti-main-diagonal sets (perpendicular to main diagonal), and vice versa.
Another operation you didn't include is a kind of "shear" (my term) - keep the first column the same, shift the 2nd column down by 1, and shift the 3rd column down by 2 (equivalent to up by 1).  This maintains the column-sets, but the old row-sets become the new main-diagonal sets, the old main-diagonal-sets become the new anti-main-diagonal-sets, and the old anti-main-diagonal-sets become the new row sets.  E.g. 
\begin{pmatrix} 
1 & 2 & 3 \\
4 & 5 & 6 \\
7 & 8 & 9 \\
\end{pmatrix}
becomes
\begin{pmatrix} 
1 & 8 & 6 \\
4 & 2 & 9 \\
7 & 5 & 3 \\
\end{pmatrix}
which can be further transformed into your 2nd matrix by swapping the first 2 rows.
However, I get quite confused when I think of how these combine with your other operations (especially reflections).  So I can't correctly count to 432 yet.
A: I believe the previous answers have sufficiently addressed the $3\times3$ matrix. In this post I will consider the $4\times4$ matrix below.
\begin{pmatrix}
0&1&2&3\\
4&5&6&7\\
8&9&A&B\\
C&D&E&F
\end{pmatrix}
Consider first the groups containing $0$:
$$(0,1,2,3),(0,4,8,C),(0,5,A,F),(0,7,A,D)$$
We see that $A$ appears in two of these groups. As such, there is only one position (relative to the position of $0$) in which $A$ can be placed, as shown here with $0$ in the upper left corner, where it will be fixed for the purpose of counting:
\begin{pmatrix}
0&?&?&?\\
?&?&?&?\\
?&?&A&?\\
?&?&?&?
\end{pmatrix}
Now there are $8$ ways to complete the two diagonals containing both $0$ and $A$:
\begin{matrix}
\begin{pmatrix}
0&?&?&?\\
?&5&?&7\\
?&?&A&?\\
?&D&?&F
\end{pmatrix}
&
\begin{pmatrix}
0&?&?&?\\
?&5&?&D\\
?&?&A&?\\
?&7&?&F
\end{pmatrix}
&
\begin{pmatrix}
0&?&?&?\\
?&7&?&5\\
?&?&A&?\\
?&F&?&D
\end{pmatrix}
&
\begin{pmatrix}
0&?&?&?\\
?&7&?&F\\
?&?&A&?\\
?&5&?&D
\end{pmatrix}
\\[2ex]
\begin{pmatrix}
0&?&?&?\\
?&D&?&5\\
?&?&A&?\\
?&F&?&7
\end{pmatrix}
&
\begin{pmatrix}
0&?&?&?\\
?&D&?&F\\
?&?&A&?\\
?&5&?&7
\end{pmatrix}
&
\begin{pmatrix}
0&?&?&?\\
?&F&?&7\\
?&?&A&?\\
?&D&?&5
\end{pmatrix}
&
\begin{pmatrix}
0&?&?&?\\
?&F&?&D\\
?&?&A&?\\
?&7&?&5
\end{pmatrix}
\end{matrix}
Next we will take one of these and fill in the remaining elements, first noting that the column containing both $D$ and $5$ must also contain both $1$ and $9$, and that $9$ does not appear in any group with $0$, so this column must be $(1,D,9,5)$. Likewise, the column $(?,F,?,7)$ must be $(3,F,B,7)$. The second and fourth rows can also be completed in this manner, and this leads to exactly one solution for each of the eight partial solutions:
\begin{matrix}
\begin{pmatrix}
0&?&?&?\\
?&D&?&F\\
?&?&A&?\\
?&5&?&7
\end{pmatrix}
&\implies&
\begin{pmatrix}
0&1&2&3\\
C&D&E&F\\
8&9&A&B\\
4&5&6&7
\end{pmatrix}
\end{matrix}
With $8$ solutions for each of the $4^2$ possible positions for the $0$, there are a total of $8\cdot4^2=128$ qualifying matrices.
