# Strategy in a waiting time game: fisherman's dilemma

Suppose there is a fisherman out on the lake, who repeatedly casts his line looking for fish. Let $X_k$ denote the time it takes him to catch a fish on his $k$th cast, $k \in \mathbb{N}$, where the $X_k$ are iid, sampled from some distribution function $F$, and so that that $X_k > 0$ and $\mathbb{E}X < \infty$. The fisherman knows $F$, and has control over one thing: he can choose to re-cast his line at any time. What is his optimal strategy to catch the most fish in the long run?

There are situations where it is advantageous to re-cast: if $F$ is bi-modal, with the two modes very far apart, then the fisherman will wait for the first mode to pass, and if he hasn't caught a fish, re-cast the line rather than waiting to hit the next mode.

I can characterize the distribution functions where it is sometimes advantageous to re-cast via a direct computation, though I am curious if there is a more elegant condition. If the fisherman hasn't seen a fish up to time $t$, then his expected waiting time to see a fish is

$$\mathbb{E}[X - t | X > t] = \int_{t}^\infty \frac{F(s)}{F(t)} ds,$$

where I'm using $F(x) = \mathbb{P}(X > x)$. On the other hand, the unconditional expected waiting time is

$$\mu = \mathbb{E}[X] = \int_0^\infty F(s) ds.$$

Thus, it is advantageous to re-cast our line at time $t$ if the conditional expected waiting time is larger than the unconditional waiting time, i.e. any time $t > 0$ satisfying

$$F(t) \int_0^\infty F(s) ds < \int_t^\infty F(s) ds.$$

$\Big($EDIT 2: @DaneiWeissman pointed out that this condition isn't always the right one to figure out the optimal strategy. Assuming that the right strategy is to pick a time $t$ and always re-cast at that time, the waiting time for that strategy is

$$\mathbb{E}[X|X \leq t] + \sum_{j \geq 0} F(t)^j(1-F(t)) jt = \frac{1}{1-F(t)} \int_0^t (F(s) - F(t)) ds + \frac{t F(t)}{1-F(t)}$$

$$=\frac{1}{1-F(t)}\int_0^t F(s)ds.$$

This is because we wait a geometrically-distributed number of times with failure probability $F(t)$ before there is a fish in $[0,t]$, and once this does occur, it takes time $\mathbb{E}[X|X \leq t]$ to catch it on average. So we should be instead be trying to solve

$$\frac{1}{F(t)} \int_t^\infty F(s) ds = \frac{1}{1-F(t)} \int_0^t F(s) ds, \hspace{5pt} (\star \star)$$

or perhaps just minimize the RHS over $t \in \mathbb{R}$. $\Big)$

For example, if these expressions are equal for all $t$, then differentiating with respect to $t$ yields

$$\mu F'(t) = -F(t),$$

and $F(t) = e^{-t/\mu}$ is the only distribution function on $[0,\infty)$ satisfying this ODE. So if the $X$'s are exponential then re-casting at any time never hurts or helps us. (This shouldn't surprise anyone familiar with Poisson processes, and the memory-less property of the exponential distribution!)

One can also compute directly with $F_\alpha(x) = (1+x)^{-\alpha}$, for $\alpha > 1$. This gives

$$\mathbb{E}[X - t | X > t] = \frac{t+1}{\alpha - 1},$$

while

$$\mathbb{E}X = \frac{1}{\alpha - 1}.$$

So the fisherman should recast his line at every positive time in this case!

I am interested in the times $t$ where we have equality, namely times $t$ when

$$\mathbb{E}[X-t|X > t] = \mathbb{E}[X] \hspace{10pt} (\star)$$

My questions are:

1) What possible sets can occur as solution sets to $(\star)$ or $(\star \star)$? For example, can there be a single unique solution? $N$ solutions for some $N \in \mathbb{N}$? Can it be satisfied for all $t$ in some interval?

2) Is there a nicer condition to determine whether or not there is some solution to $(\star$) or $(\star \star)$? (I wonder if there is some way to compare $F$ to an exponential distribution that would help.)

3) Can the integral equations $(\star)$ or $(\star \star)$ be reduced to a more tractable form?

4) Are there any other nice classes of distribution functions for which they can be solved explicitly?

5) How would one make sense of the case where $\mathbb{E}X = \infty$? The equations $(\star)$ and $(\star \star)$ don't make sense anymore. Should the fisherman ever re-cast in this case?

Also, I am curious if this question has been studied in any other context: it seems like a very natural setup to me!

P.S. There are many simple possible variants on the original question. I wonder what would happen if one imposed a time-cost on re-casting the line, or if the fisherman doesn't know how long has passed since he cast his line. What would the optimal strategy be?

EDIT 1: Another distribution one can prove something about is $F(t) = e^{-t^2}$, which decays faster than exponentially. We have $\mathbb{E}X = \frac{\sqrt{\pi}}{2}$, while

$$\mathbb{E}[X-t | X > t] = e^{t^2} \int_{t}^\infty e^{-s^2} ds.$$

It is straightforward to check that this function is strictly decreasing, and

$$\lim_{t \to 0^+} \mathbb{E}[X-t | X > t] = \mathbb{E}X,$$

which holds for any distribution function $F$ (since $F$ has right limits and $F(0) = 1$). Thus, in this case the fisherman should never recast his line.

• What would be the expected waiting time for the distribution where the fisherman catches the fish at some time $t_0$ with probability $1/2$ and never otherwise? – Peter Jan 23 '18 at 8:29
• @Peter That distribution has positive probability of being infinite, so of course the expected waiting time is infinite (any of the conditional expectations are also infinite for the same reason). The fisherman will obviously just wait until time $t_0$, and restart if he doesn't get a fish. Distributions with mass at infinity fall under the "what if $EX = \infty$" question -- it seems to me that there should still be an interesting strategy in this case. – J Richey Jan 23 '18 at 10:36
• This is a well known parameter in reliability engineering, called MTBF (mean time between fish). OK, coudn't help myself there, they call it failure instead, and try to maximize it while you minimize, but the math is similar. – MikeY Jan 23 '18 at 21:28
• @MikeY cool! I was thinking about it, and it seems to me that the maximization and minimization problems are similar but not equivalent. I will look into it -- do you know of any treatments of MTBF from a theoretical perspective? – J Richey Jan 24 '18 at 1:24
• I believe there's a mistake in the conclusion for the example of $F(t) = e^{-t^2}$. In this case, the hazard function is strictly increasing, so I think the fisherman should never recast, rather than always recast as stated. – Daniel Weissman Jan 27 '18 at 21:18

Let $f$ be the probability density function, $t$ be the recast time and $T$ be the resulting average catch time. We have: \begin{align} T &= \int_0^t x f(x)\ \mathrm{d}x + \left(\int_t^\infty f(x)\ \mathrm{d}x \right)(t + T) \\ T &= \frac{\displaystyle \int_0^t x f(x)\ \mathrm{d}x + t \int_t^\infty f(x)\ \mathrm{d}x}{\displaystyle 1 - \int_t^\infty f(x)\ \mathrm{d}x} \label{1}\tag{1} \end{align} The minimum is either at $t = 0$ (recast as soon as you can), $t = \infty$ (never recast), at $\frac{\mathrm{d}T}{\mathrm{d}t} = 0$ or at a discontinuity of $f$. The case $t = \infty$ is simply the expected value. For $t = 0$ we take the limit to 0 and need to use L'Hôpital's rule. $$\lim_{t \downarrow 0}\ T = \lim_{t \downarrow 0}\ \frac{\displaystyle \int_t^\infty f(x)\ \mathrm{d}x}{f(t)} = \frac{1}{f(0)}$$ Now the case $\frac{\mathrm{d}T}{\mathrm{d}t} = 0$: \begin{align} \frac{\mathrm{d}T}{\mathrm{d}t} &= \frac{ \left( \int_t^\infty f(x)\ \mathrm{d}x \right) \left( 1 - \int_t^\infty f(x)\ \mathrm{d}x \right) - \left( \int_0^t x f(x)\ \mathrm{d}x + t \int_t^\infty f(x)\ \mathrm{d}x \right) f(t) }{ \left( 1 - \int_t^\infty f(x)\ \mathrm{d}x \right)^2 } \\ 0 &= \left( \int_t^\infty f(x)\ \mathrm{d}x \right) \left( 1 - t f(t) - \int_t^\infty f(x)\ \mathrm{d}x \right) - f(t) \int_0^t x f(x)\ \mathrm{d}x \end{align} Some examples:

• The exponential distribution $f(x) = \lambda e^{-\lambda x}$ has $T = \frac{1}{\lambda}$. It's the only distribution in which $T$ does not depend on $t$.
• In a delayed exponential distribution $t = \infty$ gives the minimum value for $T$. $$f(x) = \begin{cases} e^{1-x} & x \ge 1 \\ 0 & x < 1 \end{cases}$$
• With the following distribution the minimum is at $t = 1$. $$f(x) = \begin{cases} \frac{1}{2} & x \in [0,1] \vee x \in [3,4] \\ 0 & \text{otherwise} \end{cases}$$
• This distribution has minima at $t = 1$ and $t = \frac52$: $$f(x) = \begin{cases} \frac{1}{3} & x \in [0,1] \vee x \in [\frac32, \frac52] \vee x \in [5, 6] \\ 0 & \text{otherwise} \end{cases}$$
• This gives a minimum at any $t \in [0, 1]$: $$f(x) = \begin{cases} e^{-x} & x \in [0,1] \\ \frac{1}{e} & x \in [2,3] \\ 0 & \text{otherwise} \end{cases}$$
• This gives a minimum at $t = 0$: $$f(x) = \begin{cases} e^{-x}-\frac{1}{e} & x \in [0,1] \\ \frac{2}{e} & x \in [2,3] \\ 0 & \text{otherwise} \end{cases}$$
• The following distribution has expected value $\infty$ and has a minimum at $t = 0$. $$f(x) = \frac{1}{(x+1)^2}$$

To your question if the fisherman should ever recast if the expected value of the distribution is $\infty$: Yes, he should always do that. Recasting gives a finite value for $T$ while not recasting will give $T = \infty$.