Is there such a function? Does there exist a continuous function $f:[0,1]\rightarrow\mathbb{R}$ such that for any two points P,Q on the curve, there exists a point R on the curve such that PQR is an equilateral triangle? If so, can we find a smooth one?
For bounty, scroll to bottom of page...
 A: Here is the answer to the generalized form of your question in the case of bounded sets. As it turns out, the convexity assumption is unnecessary.
Proposition. Let $A\subseteq\Bbb R^2$ be a bounded set, such that for each pair of distinct points $x,y\in A$ there is a point $z\in A$, such that $x,y,z$ are the vertices of an equilateral triangle. Then $A$ is one of the following:


*

*empty set,

*a set containing only one point,

*the set of vertices of some equilateral triangle.


Proof. Let $A$ be a set satisfying the hypotheses of the proposition. Suppose $A$ contains more than one point. We will show that then $A$ must be the set of vertices of some equilateral triangle.
First, we shall prove the proposition in the case that $A$ is closed. (In the end we shall show that the general case follows easily from this one.) So, let's assume $A$ is closed, i.e. $A$ contains all its limit points. Then $A$ is compact, so there exist points $a,b\in A$ such that $$d(a,b)=\operatorname{diam}A=\sup\lbrace d(x,y)|\text{ }x,y\in A\rbrace,$$
where $d$ is the metric in $\Bbb R^2$ and $\operatorname{diam}$ stands for diameter.
Let $c$ be a point such that $a,b,c$ are the vertices of an equilateral triangle. This exists by the hypotheses. Let $r=d(a,b)$ and let $S=\overline{K}(a,r)\cap\overline{K}(b,r)\cap\overline{K}(c,r)$, where $\overline{K}(x,r)$ denotes the closed ball centered at $x$ with radius $r$. Then $A\subseteq S$, because otherwise we would have $\operatorname{diam} A>r$ which is not the case. By the way, $S$ is a Reuleaux triangle:

We already know that $A$ contains the vertices of this Reuleaux triangle. We shall now show that this is all that $A$ contains. We shall first define some more points. Let $a'$ be the reflection of $a$ across the line through $b$ and $c$. Define $b'$ and $c'$ analogously. (As reflections of $b$ and $c$ across the lines opposite to them.) Let $$S_{a}:=S\setminus(\overline{K}(b',r)\cup\overline{K}(c',r))\\S_{b}:=S\setminus(\overline{K}(c',r)\cup\overline{K}(a',r))\\S_{c}:=S\setminus(\overline{K}(a',r)\cup\overline{K}(b',r))$$ Now, note that every point of $S\setminus\lbrace a,b,c\rbrace$ is contained in at least one of the sets $S_a,S_b,S_c$. So, to complete the proof in the case where $A$ is closed, it suffices to show that $A\cap S_a=\emptyset,A\cap S_b=\emptyset$ and $A\cap S_c=\emptyset$, which we will now do.
Actually, we will only prove the case of $S_a$. The other two can be proved by a completely symmetric argument, so we leave them to the reader. Let $x\in S_a$. Let $u(x)$ be the point that forms an equilateral triangle together with $a$ and $x$, for which the triangle $a,x,u(x)$ is oriented clockwise (i.e. negatively). Let $v(x)$ be the other point that forms an equilateral triangle with $a$ and $x$, i.e. $a,x,v(x)$ is oriented counterclockwise (positively). This defines two functions $u:S_a\to\Bbb R^2$, $v:S_a\to\Bbb R^2$. By definition, $u$ is a rotation around the point $a$ by $-\frac{\pi}{3}$, and $v$ is the rotation around the point $a$ by $\frac{\pi}3$.
By exploiting this geometry, one can now easily see that $u$ rotates the set $S_a$ outside of $S$, i.e. $u(S_a)\subseteq \Bbb R^2\setminus S$, and for $v$ the same holds: $v(S_a)\subseteq \Bbb R^2\setminus S$. But this means that for each $x\in S_a$ the only vertices that could form an equilateral triangle with $x$ and $a$ are outside of $S$, hence not in $A$. But then $x$ cannot be an element of $A$. Here's a picture showing $S_a$ (middle), and its two rotations (the Reuleaux triangle around $S_a$ is $S$, and the two rotations lie outside of $S$):

This completes the proof of the closed case.
In the general case of bounded (not necessarily closed) $A$ with at least two points, we proceed as follows. Let $\overline{A}$ be the closure of $A$. Since $\Bbb R^2$ is such a nice (i.e. first-countable) space, this means that $$\overline{A}=\lbrace x\in\Bbb R^2|\text{ there is a sequence } (x_n)_n\text{ with terms in } A\text{, such that } \lim_{n\to\infty}x_n=x\rbrace.$$ Now, clearly $A\subseteq\overline{A}$. Furthermore, $\overline{A}$ also satisfies the hypotheses of the proposition. To see this, let $x,y\in\overline{A}$. Then there are sequences $(x_n)_n$ and $(y_n)_n$ in $A$, such that $\lim_{n\to\infty}x_n=x$, $\lim_{n\to\infty}x_n=x$. But since $x_n,y_n\in A$ for each $n$, we can associate a point $z_n\in A$ to each pair $x_n,y_n\in A$ such that $x_n,y_n,z_n$ are the vertices of an equilateral triangle. Because $\overline{A}$ is compact, we can choose a subsequence of $(z_n)_n$ that converges to a point $z\in\overline{A}$. By continuity of the metric, $x,y,z$ again form an equilateral triangle. So, $\overline{A}$ indeed satisfies the hypotheses of the proposition and is therefore the set of vertices of some equilateral triangle. From this it easily follows that $\overline{A}=A$, which concludes the proof. $\square$
This proves among other things, that there is no continuous function $f:[0,1]\to\Bbb R$, whose graph contains for every pair of points a third point that forms an equilateral triangle with them. It proves even more: there is also no such discontinuous function $f:[0,1]\to\Bbb R$. Why? Suppose there is. Then it must be bounded, since, otherwise its graph would contain two points $x,y$ such that $d(x,y)>100$. There would then have to be a third point on the graph whose first coordinate would lie outside of $[0,1]$, which is not the case. But for bounded sets, the proposition applies. The same applies for any function $f:B\to\Bbb R$, where $B\subseteq\Bbb R$ is a bounded set.
The situation might be more interesting with functions $f:\Bbb R\to\Bbb R$, however. But in this case again, there is no such bounded function. (Same argument as in the previous paragraph.) It might be possible to construct something unbounded and probably horribly discontinuous, though, but I haven't thought much about that.
A: Not in the smooth case. Here we suppose that $f$ is $C^1$.
Let $s=\frac{\sqrt{3}}3=\tan{\fracπ6}.$
Suppose that for some $x_0$, $|f'(x_0)|<s$ then around $x_0$ the graph of $f$ looks like a straight line so horizontal that the third point of a equilateral triangle containing $(a,f(a))$ and $(b,f(b))$ would be of abscissa $x∈[a,b]$.
In the following picture, the case on the left is the one that is impossible.

Then $|f'|≥s$ and $f'$ is continuous so $f$ is monotonous so $f$ can't meet your criteria.
I don't know how to generalize.
A: Suppose that the set $S$ is bounded, and suppose that it has more than one point. Let $A$ and $B$ be two points in $S$. Then it must have a third point $C$ which is (another) vertex of the equilateral triangle with one side as $AB$, and by our condition of convexity the entire triangle $ABC$ must be in $S$.
Now let $D$ be the point of intersection of the perpendicular bisector of $\angle BAC$ and ${BC}$. Then there must be a point $E$ which is (another) vertex of the equilateral triangle with one side as $AD$. Let the distance of $AB$ be equal to $d$. Then, by several applications of the pythagorean theorem, we have that either the length of $CE=\frac {\sqrt 7} 2 d$ or $BE=\frac{\sqrt 7} 2 d$. Either way, this shows that the distance between points is unbounded, which is a contradiction with our hypothesis. Hence $S$ may not have more than one point.

If the sets are unbounded, it need not be all of $\mathbb R ^2$. For instance, any half-plane should do.

Here's a nice simple proof for why a continuous function is not possible. I've tried to be more laconic, and just give the idea of why it will not work, as I believe is in line with your preference (from the comments). Please let me know if you would like more details.

For any point (in the picture I place the point at the origin, but it's no matter) we cannot have any point of our graph on the lines emanating from this point at a $\pi/6$ angle with the horizontal, for if we did it would quickly follow that the only way to create an equilateral triangle would be to place a point above/below one of these points, and thus $f$ would not be a function. By the intermediate value theorem, this means that we cannot place any points in the shaded area, for in order to create an equilateral triangle with the point at the origin and a point in the shaded area we would need to place a point outside the shaded area, and at some point $f$ would have to cross the line. Now place a point anywhere in the non-shaded area. Immediately, from our discussion earlier, we know that $f$ cannot cross the dotted lines emanating from this point either, and furthermore the point needed to create an equilateral triangle will lie in between these lines, which we know (by the intermediate value theorem) is not a point which $f$ can reach. Hence, such a continuous function is impossible.

Also, my proof at the beginning, that a bounded set $S$ which is convex is not possible, does not really deal with showing that $S$ cannot be between a circle and a triangle, it is much simpler than that. Read it again, and note that all it shows is that, given two points $A, B$ with distance $d$, we are guaranteed that there are points $C, D$ which have distance $\frac {\sqrt 7} 2 d$. The reason this shows that the set $S$ cannot be bounded is that if it were bounded by some ball with diameter $h$, we are guaranteed that there exist points in $S$ with a distance greater than $h$. We simply pick any two points $A, B$ with distance $d$, and then repeat the process above $k$ times, where $\left(\frac {\sqrt 7} 2\right) ^k d>h$. Therefore $S$ cannot be bounded. Note that nowhere do we assume anything about bounding $S$ by the smallest ball possible, or choosing points yielding the maximum distance because (in particular, if $S$ were open) such points may not exist.
A: Edit: The bounty relates to this post. Basically, fill in all the details. I'm doing this because I am interested in the problem, however I don't have time atm to investigate.
There is no convex subset of R^2 other than:


*

*(The empty set)

*One point


which is bounded and has the property that, given any two points in the set, the 3rd point is in the set. This can be proven as such:


*

*Suppose there is such a convex bounded set in R^2 (i.e. we can stick a closed circle (i.e. closed ball) round it)

*Stick the smallest possible closed circle/closed ball round it.

*Take the two points in the set which have max distance.

*If there is a 3rd point in the set then the set must be a convex shaded region between an equilateral triangle and the circle.

*An equilateral triangle fails (think about the perpendicular bisector).

*Circle fails (diameter)

*Anything in between fails (the perpendicular from the equilateral triangle is actually outside the circle by construction (I think)).


Contradiction. Shaded concave sets... part of the bounty.
Now to unbounded sets in R^2...
We can have an unbounded set that works. E.g. take a small straight line and "extend it" by taking two points on the line and filling up the equilateral triangles. I don't know what types of sets we will get but they will be crazy and whether or not they fill up R^2 I don't know...
Still I can't work out if a cts function R-->R exists. Or if such a function R-->R exists at all. if we fill up the equilateral grid and then shift it slightly is that a function?
Bounty is in response to this question
