If I understand you right, then this may help, though it is far from a complete formula:
If we consider that a triangle is formed by the centre point, the point of intersection and the centre of the top line cd, and that the angle of the line is measured from that centre point, then the length of the line is the hypotenuse of an equilateral triangle formed by these three points.
If we now assume that the height of the rectangle is the adjacent, we can calculate the length of the hypotenuse using trigonometry as h = a / cos(t) where t is the angle of the line and o is half the rectangle's height. (Depending on software you may need to take an absolute value.)
We can repeat this formula with the width of the rectangle using cosine, hence the hypotenuse according to the width is h = o / sin(t) where a is half the rectangle's width.
The actual length of the hypotenuse must be whichever of these two values is lower.
There are other methods, such as calculating the intersection point of the line with each line that makes up the rectangle and taking the closest to the point of origin, but I think this is the more efficient.