We can get a similar result to the proposition you mention, if we assume the diagram is connected and non-empty.
Proposition. Let $I$ be a connected and non-empty category and let $\mathcal{C}$ be some category that has limits of type $I$. Fix some object $C$ in $\mathcal{C}$. Then $\mathcal{C}/C$ has all limits of type $I$ and they are calculated in the same way as in $\mathcal{C}$, in the sense that the forgetful functor $U: \mathcal{C}/C \to \mathcal{C}$ preserves limits of type $I$.
Proof. Let $F: I \to \mathcal{C}/C$ be some diagram. Denote by $U: \mathcal{C}/C \to \mathcal{C}$ the forgetful functor. Then as you already noted, we have a limiting cone $\lim UF$ in $\mathcal{C}$ with projections $p_i: \lim UF \to UF(i)$ for each object $i$ in $I$.
Now let $i$ be any object in $I$, then $F(i)$ is an object in $\mathcal{C}/C$, so it is some arrow $f_i: UF(i) \to C$ in $\mathcal{C}$. Define $\ell: \lim UF \to C$ as $\ell = f_i p_i$. This does not depend on the choice of $i$, which follows from the assumption that $I$ is connected. (This is the point where I hoped to draw a diagram, but I cannot make it work properly. So if someone else can, please do! In the meantime, try drawing it yourself on a piece of paper.) To see this, let $j$ be some object in $I$. There is a sequence of arrows between $UF(i)$ and $UF(j)$. For every step $k$ in this sequence we have a projection $p_k: \lim UF \to UF(k)$ and an arrow $f_k: UF(K) \to C$, such that everything commutes and $i$ and $j$ really give the same arrow $\ell$.
Now we do find a good candidate for the limit in $\mathcal{C}/C$, namely $\ell: \lim UF \to C$ together with the same set of projections $p_i$. This does indeed form a limit. Let $d: D \to C$ together with projections $q_i$ be some cone of $F$ in $\mathcal{C}/C$. Then $D$ together with $q_i$ forms a cone in $\mathcal{C}$. So there is an induced morphism of cones $u: D \to \lim UF$. Now we only need to check that $u$ is indeed an arrow in $\mathcal{C}/C$ as well. Let $f_i: UF(i) \to C$ be some object in the diagram of $F$, then because $q_i$ is an arrow in $\mathcal{C}/C$:
$$
d = f_i q_i,
$$
and since $u$ is a morphism of cones we have $q_i = p_i u$, so
$$
f_i q_i = f_i p_i u,
$$
finally by the definition that $\ell = f_i p_i$:
$$
f_i p_i u = \ell u.
$$
So summing up we have indeed
$$
d = f_i q_i = f_i p_i u = \ell u,
$$
as required. QED.
If the diagram is not connected, or if it is empty, we have no hope of the above proposition being true in general. Even if we assume $\mathcal{C}$ to have all limits. Consider the following two examples.
Example 1. No matter what category $\mathcal{C}$ and object $C$ we start with, the category $\mathcal{C}/C$ always has a terminal object and it is given by $Id_C: C \to C$. So if $\mathcal{C}$ already had a terminal object $1$, and we take $C$ to be non-terminal, then the forgetful functor does not preserve the terminal object.
Example 2. Let us consider $\mathbf{Set}$, the category of sets. Let us consider the set $\mathbb{N}$ of natural numbers, together with the subsets $E$ and $O$ of even and odd numbers respectively. We can naturally find $E$ and $O$ in $\mathbf{Set} / \mathbb{N}$ as well, by just considering the inclusions $E \hookrightarrow \mathbb{N}$ and $O \hookrightarrow \mathbb{N}$. The product of $E \times O$ in $\mathbf{Set}$ is just their cartesian product (with the obvious projections). The product in $\mathbf{Set} / \mathbb{N}$ does exist, but this is the empty set (with the empty function to $\mathbb{N}$)! This last part will be clear in a bit, when we prove that products in $\mathbf{Set} / \mathbb{N}$ are given by pullbacks in $\mathbf{Set}$ (so in this case, by the intersection $E \cap O$).
If we are just interested in $\mathcal{C}/C$ being complete, we have the following result.
Proposition. If $\mathcal{C}$ is complete, then so is $\mathcal{C}/C$.
This result does (implicitly) appear in most books about topos theory. When proving that for any topos $\mathcal{E}$ the slice topos $\mathcal{E}/X$, by some object $X$ from $\mathcal{E}$, is again a topos, one has to show that $\mathcal{E}/X$ is complete (although, technically this is about being finitely complete, but it easily generalises). This part of the proof only uses completeness of $\mathcal{E}$. For example, a proof can be found in Sheaves in Geometry and Logic by MacLane and Moerdijk, at the start of theorem IV.7.1. I will present a (sketch of a) proof here as well, so we can link it to the proposition at the start of this answer.
Proof. As mentioned in example 1 above, the category $\mathcal{C}/C$ always has a terminal object. By the proposition at the start of this answer, $\mathcal{C}/C$ has equalisers (and they are in fact 'the same' as in $\mathcal{C}$). So all we need to check is products. So let $(A_i \to C)_{i \in I}$ be a non-empty set of objects in $\mathcal{C}/C$. Form their wide pullback $P$ in $\mathcal{C}$. There is only one arrow $P \to C$ to be considered, and this will be the desired product in $\mathcal{C}/C$ (check this!). We have now shown that $\mathcal{C}/C$ has all small products and equalisers, so it is complete. QED.
We have essentially obtained a way to calculate limits in $\mathcal{C}/C$. For any diagram $F: D \to \mathcal{C}/C$ we obtain a diagram $F'$ in $\mathcal{C}$ by just 'forgetting' that we lived in $\mathcal{C}/C$. So I do not mean to just apply the forgetful functor here, because we want to keep all the arrows to $C$ in our diagram $F'$ (another way to describe this would be to apply the forgetful functor, and then add all the arrows to $C$ back in). Now we calculate the limit $\lim F'$ of $F'$ in $\mathcal{C}$. Since $C$ was in the diagram $F'$, we have a projection $\lim F' \to C$ and this will be the limit in $\mathcal{C}/C$.
The connection with the propisition at the start of this answer is that if $F$ is non-empty connected, we do not need to keep $C$ in the diagram to make things work.
