A $3\times 3$ matrix is positive semidefinite if and only if all of its $2\times 2$ submatrices are positive semidefinite$?$ Is it true that a $3\times 3$ Hermitian matrix, $A$, is positive semidefinite if and only if all of its $2\times 2$ submatrices are positive semidefinite$?$  
First consider the ordered set of projectors that project out a single axis 
$$P=\left(I-e_1e_1^T,I-e_2e_2^T,I-e_3e_3^T \right)$$
I can write the identity matrix as the following:
$$I = \frac{1}{2} \left(P_1+P_2+P_3 \right) $$
Now the condition in order for my 3x3 matrix, A, to be semidefinite is:
$$\forall x \in \mathbb{R}^3: x^TAx \ge0$$
Inserting the identity matrix on both sides:
$$\forall x \in \mathbb{R}^3: x^TAx=x^TIAIx \ge0$$
and expanding out my definition of the identity.
$$\forall x \in \mathbb{R}^3: x^TAx=\frac{1}{4}x^T\left(P_1+P_2+P_3 \right)A\left(P_1+P_2+P_3 \right)x \ge0$$
Expanding the product we get:
$$\forall x \in \mathbb{R}^3: x^TP_1AP_1x+x^TP_1AP_2x+x^TP_1AP_3x+x^TP_2AP_1x+x^TP_2AP_2x+x^TP_2AP_3x+x^TP_3AP_1x+x^TP_3AP_2x+x^TP_3AP_3x\ge0$$
Exploiting the hermiticity of A and the projection matrices we can rewrite the above as:
$$\forall x \in \mathbb{R}^3: x^TP_1AP_1x+2x^TP_1AP_2x+2x^TP_1AP_3x+x^TP_2AP_2x+2x^TP_2AP_3x+x^TP_3AP_3x\ge0$$
Now I believe that through the various projections each term in this sum can be independently varied with particular choices in the variation of X.  Since each term in the sum is independent of the others, the inequality distributes over each term in the sum.
$$\leftrightarrow \forall x \in \mathbb{R}^3:x^TP_1AP_1x \ge0 \cap x^TP_1AP_2x \ge0 \cap x^TP_1AP_3x \ge0 \cap x^TP_2AP_2x \ge0 \cap x^TP_2AP_3x \ge0 \cap  x^TP_3AP_3x \ge0$$
Since each term above is a 3x3 matrix with a 2x2 submatrix and the rest of the entries zero, this implies that every 2x2 submatrix of A is positive semidefinite if and only if A is positive semidefinite.  
Is my reasoning sound or is the sleep deprivation kicking in?
 A: Consider a perturbed $3 \times 3$ non positive definite matrix that has all $1$ on the diagonal and all $-1 + \epsilon$ on the off-diagonal, for some small $\epsilon > 0$. Can you see how the determinant of this matrix is negative (so not positive semi-definite), and yet every time you choose two indices $i,j$ for the two rows and the same two indices for the two columns, you get a positive definite $2 \times 2$ matrix? Or do you mean something else by "all $2 \times 2$ submatrices are positive definite"? It would seem strange to not choose the same indexed rows as indexed columns for a matrix when trying to show something like this, but let me know if I'm wrong.
As a side note, if you're wondering how to come up with an example like this, I have a bit of stats background so I think of $n \times n$ symmetric positive definite matrices as being equivalently defined as "covariance matrices", where entry $a_{ij}$ in the matrix defines the covariance (scaled correlation) between variables $i$ and $j$. If variables $i,j$ are strongly negatively correlated, and so are variables $i,k$, then variables $j,k$ should have at least some positive correlation, by composition of correlation trends. So a non-covariance symmetric $3 \times 3$ matrix can easily be obtained by setting all diagonal entries to a positive constant and then setting all off-diagonal entries to be nearly as large negative constants.
