Direction 1: $A\succeq \mathbf 0 \implies \det\big(Z_k\big)\geq 0$
(where $Z_k$ refers to an arbitrary $k\times k$ principal submatrix).
proof:
Let the eigenvalues of $A$ be given as $0\leq \lambda_n\leq \lambda_{n-1}\leq ....\leq \lambda_1$
$Z_k := S^T A S$
where
$S:= \bigg[\begin{array}{c|c|c|c|c|c|c}
\mathbf e_{\sigma_{(1)}} & \cdots & \mathbf e_{\sigma_{(k)}}
\end{array}\bigg]$
(i.e. use appropriate standard basis vectors to 'grab' the desired principal submatrix)
The eigenvalues of $Z_k$ (Cauchy) interlace those of $A$ thus for all $j\in \{1,2,...,k\}$
$0\leq\lambda_n\leq \lambda_j^{(Z_k)}\implies Z_k\succeq \mathbf 0$
hence $0\leq \prod_{j=1}^k\lambda_j^{(Z_k)}=\det\big(Z_k\big)$
(Alternative proof not using Interlacing: consider that any principal submatrix of a symmetric PSD matrix must be symmetric PSD, by a direct quadratic form argument, and thus the principal submatrix can't have negative eigenvalues and hence $\det\big(Z_k\big) \geq 0$.)
Direction 2: all $\det\big(Z_k\big) \geq 0 \implies A \succeq \mathbf 0$
proof: use strong induction on $n$
Base case:
The criterion is obvious for $n=1$.
Inductive case:
For $m\in \big\{1,2,...,n-1\big\}$: we know this is true for all $m \times m$ real symmetric matrices but need to show it is true for $n\times n$ real symmetric matrices .
Let $\text{rank}\big(A\big) = r\lt n$
Then $A$ is congruent to a particularly nice matrix. I.e.
$W^T A W = \begin{bmatrix} C_{r\times r} &\mathbf {0}\\ \mathbf {0}& \mathbf {0}_{n-r \times n-r} \end{bmatrix}$
where $C_{r\times r}$ is a principal submatrix of $A$ and $C_{r\times r}\succeq \mathbf 0$ by application of induction hypothesis
$\implies W^TA W\succeq \mathbf 0$.
Ref: Prove the existence of a principal submatrix of order $r$ in $M\in\Bbb F^{n\times n}, M=-M^T,\ \operatorname{rank}(M)=r$
Since $A$ and $W^T AW$ have the same signature, we know that $A\succeq \mathbf 0$.
= = = =
Finally, consider $\text{rank}\big(A\big) = n$
We have $\det\big(A\big)\geq 0$ and since A is non-singular this means $\det\big(A\big)\gt 0$
Consider the leading $n-1\times n-1$ submatrix, i.e. $Z:=S^T A S$ with
$S:= \bigg[\begin{array}{c|c|c|c|c|c|c}
\mathbf e_{1} & \cdots & \mathbf e_{n-1}
\end{array}\bigg]$
Applying Cauchy Interlacing we have
$\lambda_{n}\leq \lambda_{n-1}^{(Z)}\leq \lambda_{n-1}\leq \lambda_{n-2}^{(Z)} \leq \lambda_{n-2}\leq ... \leq \lambda_{1}^{(Z)}\leq \lambda_1$
By induction hypothesis we know $Z\succeq \mathbf 0\implies \lambda_i\geq \lambda_{n-1}^{(Z)}\geq 0$ for $i\in \{1,2,..,n-1\}$. And since $0\lt \det\big(A\big)=\lambda_1\cdot ...\cdot \lambda_{n-1}\cdot \lambda_{n}$ we know $\lambda_n\gt 0\implies A\succeq \mathbf 0$
(Alternative finish for $\text{rank}\big(A\big) = n$ case without using Interlacing. The leading $n-1 \times n-1$ principal submatrix $Z \succeq \mathbf 0$ by induction hypothesis and $\det\big(A\big)\gt 0$. Thus $A$ has an even number of negative eigenvalues; and if that number is non-zero we contradict the fact that $Z\succeq \mathbf 0$; this is mechanically the same as the standard proof for Sylvester's Criterion for PD matrices, see e.g. here: Characterization of positive definite matrix with principal minors )