Find a circle on a plane defined by three points in 3D and tangent at two of its points?

Question

Given: Three points $A~(x1,y1,z1)$, $B~(x2,y2,z2)$, and $C~(x3,y3,z3)$. Points A and C are equidistant from B i.e. $AB = BC = d$.

Aim: Find the circle (particularly its portion shown in magenta color) which lies on the plane which is formed by the three points and tangent at the points A and C. I am able to solve the problem for the 2D case but stuck in 3D case. Here is my algorithm (I'm stuck at points 4,5 and 7):

1. Find the equation of the plane formed by three points: $Ax + By + Cz + D = 0\\ A = \begin{vmatrix} 1 & y1 & z1\\ 1 & y2 & z2\\ 1 & y3 & z3 \end{vmatrix}, B = \begin{vmatrix} x1 & 1 & z1\\ x2 & 1 & z2\\ x3 & 1 & z3 \end{vmatrix}, C = \begin{vmatrix} x1 & y1 & 1\\ x2 & y2 & 1\\ x3 & y3 & 1 \end{vmatrix}, -D = \begin{vmatrix} x1 & y1 & z1\\ x2 & y2 & z2\\ x3 & y3 & z3 \end{vmatrix}$
2. Find equation of line AB: $(x1,y1,z1) + t(x2-x1,y2-y1,z2-z1)$
3. Similarly equation of line BC: $(x2,y2,z2) + t(x3-x2,y3-y2,z3-z2)$
4. Find the equation of line which is perpendicular to AB at point A and which also lies on the plane. (I'm stuck to find this line) I understand that the dot product of the two lines must be zero. But the constraint that it must also lie on the plane and include the point.
5. Similarly find the equation of line which is perpendicular to BC at point C and which also lies on the plane. (I'm stuck to find this line like step 4)
6. The point of intersection of lines formed at step 4 and 5 gives the center of the circle $o~(cx,cy,cz)$ of radius $r=\sqrt{(cx-x2)^2+(cy-y2)^2+(cz-z2)^2}$ (???)
7. How to find the circle with center (o $cx,cy,cz$) and radius $r$ which also lies on the plane?

In 2D case, I could find the slope (m) and then the perpendicular would have slope of ($\frac{-1}{m}$) which lead to easy solution, but I'm stuck in steps 4, 5 and 7 above.

Answer 1

Useful bit of information: the cross product of two vectors is perpendicular to both of them.

1. $n=(A-B)\times(C-B)$ is the normal of your plane. This is the vector $(A,B,C)$ formed by three of the coefficients from the equation of the plane in your step 1.
2. $v=n\times(A-B)$ is the direction within the plane and perpendicular to $AB$. Likewise $w=n\times(C-B)$ for the direction perpendicular to $BC$. So a point on these lines coud be represented as $A+\lambda v$ resp. $C+\mu w$. This should help you with steps 4 and 5.
3. To intersect these lines, you can simply write the equation for a point which lies on both of them: $D:=A+\lambda v=C+\mu w$. Then solve for $\lambda$ and $\mu$. There are actually three equations, one for each coordinate, and two variables, so this is over-determined. You can concentrate on two of the three coordinates to solve this, or use some form of pseudo-inverse if you want to avoid special cases. The result is the center of the circle.
4. If you have the center, and have the radius, how can you describe the circle itself? Well, one possible description would be using center, radius and normal vector $n$. Another would be a parametric description, for which you'd need to find two orthonormal vectors within the plane. $e_1=(A-B)/\lVert A-B\rVert$ could be one such vector, and $e_2=(n\times e_1)/\lVert n\times e_1\rVert$ could be the other. Then the circle would be $D+r\cos\theta e_1+r\sin\theta e_2$ or something like that.