Orientation is a concept meant for ordered bases of a vector space. If we have the ordered basis $(b_1,\dots, b_n)$ we say that it has a certain orientation. And we say that if we exchange two of the basis vectors to get, for instance, $(b_k,b_2,\dots,b_{k-1},b_1,b_{k+1},\dots,b_n)$, then we change the orientation of the basis. If we switch two more basis elements, then we change back to the original orientation (though not necessarily to the original ordered base). This way, we can define two orientations for an ordering of a given base. One we call left-handed, the other right-handed.
Now the determinant has this concept of orientation baked into its definition. Part of its definition is that if we exchange two columns of a matrix, then we flip the sign of its determinant (more formally, it is antisymmetric). What's also baked into it is that if a linear map with matrix representation $M$ (with respect to the ordered base above) is injective, then its determinant is non-zero. So if we take any matrix with non-zero determinant, then it maps an ordered base to a different ordered base: $(b_1,\dots,b_n)\mapsto (Mb_1,\dots,Mb_n)$. Now we said earlier that, for instance, the ordered base $(Mb_k,Mb_2,\dots,Mb_{k-1},Mb_1,Mb_{k+1},\dots,Mb_n)$ should have the opposite orientation of $(Mb_1,\dots,Mb_n)$, since we exchanged two of the basis vectors. And it miraculously (or not so miraculously after all ;) ) happens that we get to this other ordered base by exchanging two columns of $M$ and applying the new matrix to $(b_1,\dots, b_n)$. But at the same time we defined that the determinant of the matrix flips signs if we do that! So the sign of the determinant is a good way to define an orientation for all ordered bases: Let $R:=(b_1,\dots,b_n)$ be an ordered base , our reference base, which we define to be right-handed. Then we say that any other ordered base $B$ is right-handed if the determinant of the matrix transforming $R$ to $B$ is positive, and left-handed if the determinant is negative.
So to summarize: The sign of a matrix is the definition of what orientation preserving means. Positive sign is orientation preserving, negative sign is orientation inverting.