This is an exercise in Category Theory In Context by Riehl.
Here is the quote in which she describes the proof strategy:
To show that $U : C^A \to C^{ob A}$ strictly creates all limits and colimits, we must show that for any $F : J → C^A$ the $ob A$-indexed family of objects $\lim Fj(a)$ extends to a functor on $A$ valued in $C$. The universal property of the limit construction is used to define the action of this functor on morphisms in A. The uniqueness statement in this universal property then implies that this construction is functorial.
I am not quite sure where to begin. To show that $U$ strictly creates limits, we let $F : J \to C^A$ be a diagram. Then by a previous result, $UF$ has a limit in $C^{ob A}$ whose limit cone is comprised of morphisms (natural transformations) $\lim UF \to UFj$; on the level of components this cone is a commuting prism with a layer for each $a \in A$, but no arrows between layers as the category $ob A$ is discrete.
My precise point of confusion is determining how to take this data and lift it to a cone over $F$; we have a bunch of cones formed by evaluating everything at $a$, none of which are connected to each other by arrows, so there is no constraint I can see on the legs of each cone that should allow us to lift up to legs that commute with the arrows in $F$'s image.
If I have made some grievous error, please let me know, but otherwise I would love a hint or some explanation about the details of the construction.