Without the unit-length segment--that is, without something to compare the first segment to--its length is entirely arbitrary, so can't be valued, so there's no value of which to take the square root.
Let the given segment (with length x) be AB and let point C be on ray AB such that BC = 1. Construct the midpoint M of segment AC, construct the circle with center M passing through A, construct the line perpendicular to AB through B, and let D be one of the intersections of that line with the circle centered at M (call the other intersection E). BD = sqrt(x).
AC and DE are chords of the circle intersecting at B, so by the power of a point theorem, AB * BC = DB * BE, so x * 1 = x = DB * BE. Since DE is perpendicular to AC and AC is a diameter of the circle, AC bisects DE and DB = BE, so x = DB^2 or DB = sqrt(x).
edit: this is a special case of the more general geometric-mean construction. Given two lengths AB and BC (arranged as above), the above construction produces the length BD = sqrt(AB * BC), which is the geometric mean of AB and BC.