I practice identifying adjoint functors on simple categories.

Now I came across a case, where it seems, I have a counit-unit adjunction, but not a hom-set adjunction. Is this possible?

For the concrete example:

$$F: \mathcal{C} \to \mathcal{D},~~ G: \mathcal{C} \leftarrow \mathcal{D} \\ a, b, b', ~h: a \to b, ~h': b' \to b \in \mathcal{C} \\ Fa, Fb, ~Fh: Fa \to Fb \in \mathcal{D} \\ Fb' = Fb, ~Fh' = 1_{Fb}$$

I can work out $\eta$ and $\epsilon$:

$$\eta_a = 1_a, ~~\eta_b = 1_b, ~~\eta_{b'} = h' \\ \varepsilon_{Fd} = 1_{Fd}, ~~\varepsilon_{Fe} = 1_{Fe}$$ And they fulfill triangle identities.

But I can't work out the bijections for $\Phi_{Fa,b'}: Hom(GFa,b') \cong Hom(Fa,Fb')$.

$Hom(Fa,Fb')$ contains $Fh$, but $Hom(GFa,b')$ seems to be the empty set.

In your adjunction, $G$ is the right adjoint and $F$ is the left adjoint, so the bijection of Hom-sets you ask for is incorrect. There would instead be a bijection $\mathrm{Hom}(a,GFb')\cong\mathrm{Hom}(Fa,Fb')$, and indeed there is.