Actually one can say more: if $d'$ is a homotopy inverse of $d$ and (omitting the $\circ$'s) $h:d' d\simeq 1, k: d d' \simeq 1$, then there is a homotopy inverse $f'$ of $f$ with homotopies
$h':f' f \simeq 1, k': f f' \simeq 1$ such that $f'$ extends $d'$, $h'$ extends $h$, while $k'$ extends the composite of the homotopies
$$ d' d = d' d 1_A \simeq d'd d' d \simeq d'1_Bd \simeq 1_A$$
determined by $h,k$.
Comments: 1) This result was arrived at by generalising the classic proof that a homotopy equivalence of spaces induces an isomorphism of homotopy groups. (The point is that the homotopy equivalence is not given as a homotopy equivalence of spaces with base points.) A crucial technique in the proof is the operation of the fundamental groupoid on the higher homotopy groups. When the base point in a sphere is replaced by a cofibration $i: A \to X$ one replaces a fundamental groupoid of $Y$ by the track groupoid $\pi_1 Y^A $ of homotopy classes of homotopies $A \to X$, and let this act on the family of homotopy classes $[(X,i),(Y,u)]$ for all $u:A \to Y$.
2) The advantage of stating the precise control of the homotopies is that it makes it easy to glue homotopy equivalences along the spaces $A,B$. That was how I found the Gluing Lemma for homotopy equivalences for the 1968 edition of Topology and Groupoids, and which is in Section 7.4 of T&G; also it is easy to generalise from gluing two spaces to gluing $n$ spaces (Exercise 1 of Section 7.4).
3) The curious composite of homotopies is related to the elementary lemma in category theory that a right inverse of an isomorphism is itself an isomorphism. This can be applied to homotopy equivalences of spaces instead of isomorphisms, but the relevance comes when you try to identify the homotopies. I expect that the above statement on $k'$ can be recovered from Peter May's proof.
4) I don't think it is possible to get in general the homotopy $k'$ to extend $k$ but I do not have a counterexample. (One has been given in the dual fibration case.)