You know that after $n $ games, team A has won $k $ of those games with
$$P = {n\choose k}0.6^k\cdot0.4^{n-k}$$
(Why?)
Similarly, if they play $n $ games, then team B won $k $ games with a probability of
$$P = {n\choose k}0.4^k\cdot0.6^{n-k}$$
You want to know the number of games that is most likely to be enough to determine the winner. If a team must win 3 out of 5, it suffices to win the first 3 games, or win 3 out of the first 4, or win 3 out of the first 5.
Please note that the probability of a team winning 3 out of 4, but without winning the first 3, is a bit different than the probability of winning 3 out of 4.
What you want to check now is the probabilities of
Either team winning 3 games in a row
Either team winning 3 out of 4 games without having 3 wins in the first 3 games
Either team winning 3 out of 5 games without having 3 wins in the first 4 games
One of those 3 situations will be the most likely. That is your answer.