Shift your coordinates so that the point of the angle is the origin. Normalize the other points (so we're replacing all other points by ones that are a unit distance from the "origin". Say the cone is made by $O,A,B$ (in the new coordinates) and the point to find is $C$. If $A\cdot C$ and $B\cdot C$ are greater than $A\cdot B$, then it's between them.
EDIT: formula, with $O,A,B,C$ as the original, non-shifted points:
$$D_{AB}=\frac{(A_x-O_x)(B_x-O_x)+(A_y-O_y)(B_y-O_y)}{\sqrt{(A_x-O_x)^2+(A_y-O_y)^2}\sqrt{(B_x-O_x)^2+(B_y-O_y)^2}}$$
If $D_{AB}<D_{AC}$ and $D_{AB}<D_{BC}$, then $C$ is in the angle. Otherwise, it's outside. The matrix method given on the stackoverflow page is also good, and may be easier. In this, $A,B,C$ have already been shifted.
$$C_x=\alpha A_x+\beta B_x$$
$$C_y=\alpha A_y+\beta B_y$$
if $\alpha$ and $\beta$ are both positive, then it's in the interior.
$$\alpha=\frac{C_yB_x-C_xB_y}{A_yB_x-A_xB_y}$$
$$\beta=\frac{C_yA_x-C_xA_y}{B_yA_x-B_xA_y}$$
My only suggestion is that the algorithm hasn't been implemented properly, possibly because of some subtle language/code issue. I'd advise you to post your written code on the stackoverflow question, it's harder to help without it.