Optimization path avoiding a set of points I have a random polygon, convex or non-convex, defined by its vertices and two random points outside of the polygon (A and B) all of them defined in ${\rm I\!R}^{2}$, how can I get an optimized path, the shortest path, from the point A to point B rounding the polygon, i.e., the path cannot go through the polygon, it is important to note that the polygon is defined as a set of points, its vertices, where the first one and last one are equals and counter-clockwise, i.e., joining the points you get a counter-clockwise polygon.
 A: I would think you can use the convex hull generated by $A$, $B$ and the polygon.
Clearly then $A$, $B$ are located on the boundary of the convex hull. 
Then $A$, $B$ dissect the boundary in 2 parts. Choose the part which is shorter than the other.
A: I’ll assume the following


*

*An “optimized path” is the shortest path in the Euclidean sense

*The polygon is equal to the convex hull of the given vertices


By rotation and scaling, we can assume that $A$ and $B$ lie on the $x$-axis, with $A$ strictly to the left of $B$. Let $V$ denote the given set of vertices. 
Partition $V$ into $V^+$ and $V^-$, the set of vertices that lie above and below the $x$-axis, respectively.
Sort the vertices in $V^+$ by increasing $x$-coordinate (ties are broken by increasing $y$-coordinate). Label these vertices $v_0,v_1,\dots,v_{n+}$. Starting at $v_0$, move along until you find the vertex $v_k$ such that every vertex $v_0,\dots,v_{k-1}$ lies below the line connecting $A$ and $v_k$, but $v_k$ lies strictly above the line connecting $A$ and $v_{k+1}$. The first leg of the path is the straight line connecting $A$ to $v_k$.
Next, starting at $v_k$, continue moving through the list until you find a vertex $v_\ell$ such that every vertex $v_{\ell+1},\dots,v_{n+}$ lies strictly below the line connecting $v_\ell$ and $B$, but $v_\ell$ lies strictly above the line connecting $v_{\ell-1}$ to $B$. The next section of the path is the path from $v_k$ to $v_{k+1}$ to ... to $v_\ell$.
The final segment of the path is the straight line connecting $v_\ell$ to $B$.
Repeat the above for $V^-$, and pick the shorter of the two resulting paths. 
