Homotopic maps induce isomorphic pullbacks of a principal bundle. How functorial can this be?

More precisely, I'm trying to show that the groupoid $\mathscr{B}G(X)$ of principal $G$-bundles over $X$ and isomorphisms is equivalent to $\Pi_1(BG^X)$. It seems like the right direction to try to construct my functor is $\Pi_1(BG^X) \rightarrow \mathscr{B}G(X)$, since given an actual $G$-bundle over $X$ I'm not sure how I should expect to make a choice of map $X\rightarrow BG$ (as opposed to just a choice of equivalence class up to homotopy).

So, to an object $(f:X\rightarrow BG)\in \Pi_1(BG^X)$ we of course associate $f^*EG\in \mathscr{B}G(X)$. Then, given a morphism $([H]:X\times I \rightarrow BG)\in \Pi_1(BG^X)(f_0,f_1)$ (i.e. an equivalence class up to homotopy of paths from $f_0$ to $f_1$ in $BG^X$) We then use the lifting property, which can be done $G$-equivariantly, to get an isomorphism $f_0^*EG \rightarrow f_1^*EG$.

The problem is that unless I've got something a bit stronger this isn't going to be functorial: the composition of two homotopy-classes-of-paths in $BG^X$ won't necessarily give rise to the composite isomorphism. It seems like what I want is a connection that'll tell me what "horizontal" means, but this is definitely not part of the data I'm given.

Bonus question: This doesn't work for topological groups (let $G$ be connected and take $X=pt$). How can this be repaired -- is there a good alternative description of $\mathscr{B}G(X)$ in that case?

-
Wowwwww. Of course you're right. It's hard to believe it but I guess all the fancy talk (I didn't even say "stack"!) got in the way of good ol' Hatcher Chapter One. – Aaron Mazel-Gee Jun 24 '11 at 20:27
Feel free to post this as an answer, or I'll CW it and answer it myself. – Aaron Mazel-Gee Jun 24 '11 at 20:28
OK (and what happens for connected G and X=pt, BTW?) – Grigory M Jun 24 '11 at 20:48
$\pi_1(BG)=\pi_0(G)=1$, which would mean that a $G$-bundle over a point has no automorphisms. That's not right. – Aaron Mazel-Gee Jun 24 '11 at 22:27
Well, $\Pi_1(BG^X)$ describes not $(Bun,Iso)$ but $(Bun,\pi_0(Iso))$, I believe. Perhaps, you need $\Pi_\infty(BG^X)$ to get all $BG(X)$, in some sense. – Grigory M Jun 25 '11 at 7:16

If $G$ is discrete, $EG\to BG$ is a covering, so the lifting is unique. Since composition of liftings is always a lifting of composition, this implies functoriality.