I am confused about the difference between positive semi-definite and positive definite.
May I understand that positive semi-definite means symmetric and $x'Ax \ge 0$, while positive definite means symmetric and $x'Ax \gt 0$?
Yes. In general a matrix $A$ is called...
indefinite if it is nothing of those.
Literature: e.g. Harville (1997) Matrix Algebra From A Statisticians's Perspective Section 14.2
A great source for results about positive (semi-)definite matrices is Chapter 7 in Horn, Johnson (2013) Matrix Analysis, 2nd edition. One result I found particularly interesting:
Corollary 7.1.7. A positive semidefinite matrix is positive definite if and only if it is nonsingular.
A symmetric matrix A is said to be positive definite if for for all non zero X $X^tAX>0$ and it said be positive semidefinite if their exist some nonzero X such that $X^tAX>=0$.