rock, paper, scissors, well Everyone knows rock, paper, scissors. Now a long time ago, when I was a child, someone claimed to me that there was not only those three, but also as fourth option the well. The well wins against rock and scissors (because both fall into it) but loses against paper (because the paper covers it).
Now I wonder: What would be the ideal playing strategy for rock, paper, scissors, well?
It's obvious that now the different options are no longer on equal footing. The well wins against two of the three other options, and also the paper now wins over two options, namely rock and well. On the other hand, rock and scissors only win on one of their three possible opponents.
Moreover, the scissors seem to have an advantage to the rock, as it wins against a "strong" symbol, namely paper, while the rock only wins against the "weak" symbol scissors.
Only playing "strong" symbols is obviously not a good idea because of those two, the paper always wins, so if both players only played strong symbols, the clear winning strategy would be to play paper each time; however if you play paper each time, you're predictable, and your opponent can beat you by selecting scissors.
So what if you play only well, paper and scissors, but all with the same probability? If your opponent knows or guesses it, it's obviously undesirable to choose rock, because in two of three cases he'd lose, while with any other symbol, he'd lose only in one of three cases. But if nobody plays rock, we are effectively at the original three-symbol game, except that the rock is now replaced by the well.
Therefore my hypothesis is: The ideal strategy for this game is to never play rock, and play each other symbol with equal probability.
Is my hypothesis right? If not, what is the ideal strategy?
 A: You are quite right. Rock is dominated by Well: no matter what the opponent plays, you do at least as well playing Well as you would playing Rock, and against Rock or Well you do better. Good players will never play Rock, so the game reduces to Well, Paper, Scissors, which is isomorphic to Rock, Paper, Scissors.
A: Mixing evenly between paper, scissors, and well is indeed an equilibrium. 
Starting with Vadim's condition:
$$ p-s+(1-r-p-s)=\\-r+s-(1-r-p-s)=\\r-p+(1-r-p-s)=\\-r+p-s$$
If Rock receives no weight, we have:
$$s-(1-p-s)=\\-p+(1-p-s)=\\p-s$$
Which gives $p=s=(1-p-s)=\frac{1}{3}$
Further, Rock is dominated by any combination of the other three strategies against this mixture. Thus, any mixture of the three is indeed a best response to an equal mixture and so the equal mixture is a Nash equilibrium.  
To see there is no other equilibrium, we can use the fact that in a symmetric non-zero sum game, any strategy optimal for one player is optimal for another. Note that when rock receives weight in opponent's strategy, rock is strictly dominated by well. Thus, rock cannot be part of an equilibrium since it would imply that rock is part of an optimal strategy against a strategy that includes positive weight on rock. 
A: Here is the payoff matrix: $$\begin{matrix} &R&P&S&W\\ R&0&1&-1&1\\ P& -1&0&1&-1\\ S & 1 & -1 & 0 & 1\\W & -1 & 1 & -1 & 0\end{matrix}$$
Suppose we pick $R$ with probability $r$, $P$ with probability $p$, $S$ with probability $s$, and $W$ with probability $1-r-p-s$.  An optimal mixed strategy is indifferent to the opponent's choice.  Hence
$$ p-s+(1-r-p-s)=\\-r+s-(1-r-p-s)=\\r-p+(1-r-p-s)=\\-r+p-s$$
Unfortunately there is no solution to this.  Alas, I don't have time right now to take this further.
