Why are these two definitions of the left-adjoint to $u^p\colon PShv(D)\to PShv(C)$ equivalent? Suppose $u\colon C\to D$ is a functor between categories. Then there is a functor
$$
u^p\colon PShv(D)\to PShv(C)
$$
between the associated presheaf categories by precomposition with $u$ as it is explained in the stacks project.
One may construct a left-adjoint $u_p$ to $u^p$ in two ways:
First, one sets for $F\in PShv(C)$ and $V\in D$
$$
\begin{array}{rcl}
u_p(F)(V)&=&\underset{(YU\to F)\in \int_CF}{\operatorname{colim}} Y(u(U))(V)\\
         &=&\underset{(YU\to F)\in \int_CF}{\operatorname{colim}} Hom_{D}(V,uU)
\end{array}
$$
where $Y$ denotes the Yoneda embedding of $C$ (resp. $D$) into the $PShv(C)$ (resp. $PShv(D)$) and where $\int_CF$ is the category of elements of $F$ whose objects are natural transformations $YU\to F$ for some $U\in C$ and whose morphisms are given by $Yf\colon YU\to YU'$ for some morphisms $f\colon U\to U'$ of $C$ fitting into a commutative triangle over $F$. This construction of the left-adjoint is a left Kan extension. One has given the functor $u$ from representables and prolongs it universally to a functor from the whole category of presheaves.
Secondly, according to the stacks project, the left-adjoint $v^p$ (I just temporarily write $v^p$ for $u^p$ to distinguish from the first definition) $u_p$ is given for $F\in PShv(C)$ and $V\in D$ by
$$
v_p(F)(V)= \underset{(V\to uU)\in (I_V^u)^{opp}}{\operatorname{colim}} F(U)
$$
where $I_V^u$ is the category whose objects are morphisms $V\to u(U)$ in $D$ for some $U\in C$ and whose morphisms are given by $uf\colon uU\to uU'$ for some morphisms $f\colon U\to U'$ of $C$ fitting into a commutative triangle under $V$.
Of course, one may say that a left adjoint to a given functor $u^p$ is unique up to natural isomorphism of functors but my question is:

Is there a concrete natural isomorphism between $u_p$ and $v_p$? If yes, how is it constructed and how to show that it is an isomorphism?

 A: Concerning $v_p$:
For any ${\mathscr F}\in\text{PreShv}({\mathcal C})$ we have $$(\ast)\quad\underset{\alpha\in {\mathscr F}U}{\text{colim}}\ {\mathcal C}(-,U)\xrightarrow{\cong} {\mathscr F}.$$
$$(\alpha\in{\mathscr F}U,\beta: U^{\prime}\to U)\longmapsto {\mathscr F}(\beta)(\alpha)\in{\mathscr F}(U^{\prime}).$$
Plugging this into your second description gives
$$(\dagger)\quad v_p({\mathscr F})(V) := \underset{V\to uU}{\text{colim}}\ {\mathscr F}(U)\cong\underset{V\to uU}{\text{colim}}\ \underset{\alpha\in {\mathscr F}U^{\prime}}{\text{colim}}\ {\mathcal C}(U,U^{\prime}).$$
Concerning $u_p$: 
Note that for any $V\in{\mathcal D}$ there is a canonical isomorphism
$$\underset{V\to u U^{\prime}}{\text{colim}}\ {\mathcal C}(U^{\prime},U)\xrightarrow{\cong} {\mathcal D}(V,uU), $$
$$(\alpha: V\to u U^{\prime}, \beta: U^{\prime}\to U)\ \ \longmapsto\ \  {\mathscr F}(\beta)\circ\alpha$$
$$\gamma: V\to uU\ \ \longmapsto \ \ (\alpha := \gamma, \beta := \text{id}_{U=U^{\prime}});$$
This is a special case of $(\ast)$ applied to the presheaf ${\mathscr G}: U\mapsto{\mathcal D}(V,uU)$ on ${\mathcal C}^{\text{opp}}$: An element of ${\mathscr G}$ is a pair $(U^{\prime},\alpha)$ with $U^{\prime}\in\text{Obj}({\mathcal C}^{\text{opp}})=\text{Obj}({\mathcal C})$ and $\alpha\in{\mathscr G}(U^{\prime})={\mathcal D}(V,uU^{\prime})$, so $${\mathcal D}(V,u(-))\quad\cong\quad\underset{V\to uU^{\prime}}{\text{colim}}\ {\mathcal C}^{\text{opp}}(-,U^{\prime})=\underset{V\to uU^{\prime}}{\text{colim}}\ {\mathcal C}(U^{\prime},-).$$
Using this in your description of $u_p$, you get 
$$(\ddagger)\quad u_p({\mathscr F})(V) = \underset{\alpha\in {\mathscr F}U}{\text{colim}}\ {\mathcal D}(V,uU)\cong\underset{\alpha\in{\mathscr F}U}{\text{colim}}\ \underset{V\to u U^{\prime}}{\text{colim}}\ {\mathcal C}(U^{\prime},U)$$
Conclusion: 
Comparing $(\dagger)$ and $(\ddagger)$ we see that $v_p$ and $u_p$ are isomorphic by renaming $U$, $U^{\prime}$ and exchanging the colimits.
