What you need to do is to let $(x,y)$ be the point at which he hits the shore, stopping rowing and starting running (assuming he just hops out of his boat and takes off).
This will be controlled by the angle $\theta$ he makes. Clearly, if he goes along segment $ac$, it will take him 4 hours and 20 minutes--13miles / 3mph = 4 hrs, 20 min. This angle would be $\arccos\left(\frac{5}{12}\right)$.
If he goes along segment $ac$, this will take him 5 / 3 = 1 hr 40 min + 12 / 5 = 2 hr, 24 min, which is 4hrs, 4 minutes. Let's just call this angle 0.
The time equation is then given by $t(\theta) = 5\sec(\theta) / 3 + (12-5\cot(\theta))/ 5 $
(this may not be correct; I don't currently have access to any scratch paper to really nail this down)
Then you simply need to apply your max/min problem-solving skills. This is equivalent to Ghost's above, but his solution is somewhat simpler, and doesn't involve messy trig calculations, so I would use his--with the slight modification that you need to divide by the speed instead of multiplying: $T(p_1 \cup p_2) = \sqrt{5^2 + d^2} / 3 + (12 -d ) /5$ However, this works just as well.
