First let's describe what to do when the intersection is at a vertex $v$ of the polytope. (See the image below for the 2D version, but everything I say will apply just as well in 3D.) We assume convexity so that the centroid gives us a inward vector at any vertex.
There are some number of faces of the polytope meeting the vertex $v$, each of which spans a plane (blue and orange dashed lines).
For each face, we can choose an inward vector orthogonal to the face, because we have the known inward vector from $v$ to the centroid to compare with (black dashed vector). Namely, if $v_i, v_j$ span the plane of the face then we can take $\pm (v_i\times v_j)$ where the sign is equal to that of $(v_i\times v_j)\cdot c$ where $c$ is the vector to the centroid from $v$.
Supposing we have a ray starting outside the polytope and passing through the face at $v$, we can measure whether a ray given by vector $r$ is inward or outward with respect to that face by taking the dot product with the normal vector obtained in the previous step and checking the sign. In other words $$(r \cdot (v_i\times v_j))(c\cdot(v_i\times v_j)) > 0$$
For a ray to pass inside the polytope, it must be inwards with respect to all faces, so (3.) should be repeated for every face (but you can exit early as soon as you find an outward direction with some face).
Note that if the ray comes from inside the polytope, then instead all the signs will be outward for each face. (It will be easy to account for this.) Also, if the dot product in (3.) is ever 0 then the ray intersects along the boundary at that face, and it is not interior.
Hopefully the image makes it intuitive that the test rays starting outside the polytope begin entering the interior of the polytope precisely when they have a positive dot product with the normal vectors to each face.
As for the case when intersection is an edge $(v_1,v_2)$, then you may apply the same algorithm as above but only to the pair of faces touching the edge. To find those faces simply go to $v_1$ and pick out the faces containing $v_2$.
Finally, when the intersection is a face, you only need to check the ray is not parallel to the face itself. One way is to find an inward normal vector to the plane as above and check that the dot product does not vanish.