# Use stochastic dynamic programming to find the optimal (in terms of expected total profit) investment strategy

Question:

Three investment schemes (A, B, and C) accept investments in multiples of £3000.

Schemes A and B mature after one year, and generate returns distributed as follows:

$$\begin{array}{c|lcr} & \text{annual return on £3000 investment} & \text{Probability} \\ \hline A & 2,000 & 0.2 \\ & 5,000 & 0.8 \\ B & 0 & 0.1 \\ & 4,000 & 0.4 \\ & 6,000 & 0.5 \end{array}$$

Scheme C matures after two years, generating a guaranteed return as follows:

$$\begin{array}{c|lcr} & \text{two-year return on £3000 investment} & \text{Probability} \\ \hline C & 7,000 & 1 \end{array}$$

You have $£4,000$ to invest at the start of a 2 year period.

At the end of a year that does not bring the investment period to a close, any return from that year, together with any amount not invested that year, is available to invest for the next year. Investment may be split between the schemes as desired (provided the rule that investments must be in units of £3000 is respected).

So firstly I want to find the investment which is best over a 1 year period. I convert the tables into how much profit they produce giving

$$\begin{array}{c|lcr} & \text{annual return on £3000 investment} & \text{Probability} \\ \hline A & -1,000 & 0.2 \\ & 2,000 & 0.8 \\ B & -3,000 & 0.1 \\ & 1,000 & 0.4 \\ & 3,000 & 0.5 \end{array}$$

So over a one year period the optimal strategy (in 1,000's) for each scheme would be

$$A: 0.2(-1)+0.8(2) = 1.4 \\ B: 0.1(-3) + 0.4(1) + 0.5(3) = 1.6$$

meaning that $B$ would be the best strategy to use.

Now for the 2 year period I say that for

$$A: 0.2(-1 + f_2^*(2)) + 0.8(2+f_2^*(5)) \\ B: 0.1(-3 + f_2^*(0)) + 0.4(1+f_2^*(4)) + 0.5(3+f_2^*(6)) \\ C: = 4,000$$

This is by using the backwards induction method. Now I say that

$$f_2^*(0) = 0 \\ f_2^*(2) = f_2^*(4) = 1.6 \\ f_2^*(5) = f_2^*(6) = 3.2$$

Which gives

$$A: = 4,280 \\ B: = 3,840 \\ c: = 4,000$$

This means that the optimal strategy would be to invest in $A$ for year 1 then $B$ for year 2.

I am unsure whether what I have done is correct. I am just looking for someone to confirm whether what I have done is right.

The method I have used is backwards induction.