Can we say that a semi-positive definite matrix is always a positive definite matrix but not vice versa? If No, kindly explain.
No; in fact, the opposite holds. If a symmetric matrix $A$ is positive definite, then $x^TAx > 0$ for all nonzero $x$. If $x=0$, then $x^TAx = 0$, and so in general $x^TAx \geq 0$, and so $A$ is positive semi-definite.
Conversely, note that the zero matrix is positive semi-definite but not positive definite.
Any positive definite matrix is also positive semi-definite. This is not true in the other order. This is because if A is positive definite, then for any non-zero vector $z$ we have $z^TAz>0$ this implies $z^TAz\ge0$ which is the definition of a positive semi-definite matrix. You can see that the other way is not true because if $z^TAz=0$ then $z^TAz\ge0$ but not $z^TAz>0$