In which sense is cohomology dual to homology? Colimits in a category $C$ are called "colimits" because they are limits in the dual category $C^\mathrm{op}$.
Question: Why is singular cohomology called "cohomology"? In which way does it arise from singular homology by "dualizing" it? It is not as simple as in the example above: I don't know what $C$ is in this case.
 A: As noted in comments you work with singular (co)homology.
Both homology and cohomology arise by considering the singular chain complex first:
$$\cdots \xrightarrow{} C_{i+1} \xrightarrow{\partial_{i+1}} C_{i}\xrightarrow{\partial_{i}} C_{i-1}\xrightarrow{}\cdots$$
At this point homology is defined as $H_i(X):=\text{ker }\partial_i/\text{im }\partial_{i+1}$. Then it can be shown that $H_i$ is functorial in a covariant manner, i.e. a continuous function $f:X\to Y$ is converted to $H_i(f):H_i(X)\to H_i(Y)$ homomorphism.
With cohomology we add one more step. We take an abelian group $A$ and convert our original singular chain complex by applying $\text{Hom}(\cdot,A)$ to it. Since $\text{Hom}$ is contravariant (it reverses arrows), this gives us
$$\cdots \xleftarrow{} C^{i+1} \xleftarrow{\partial^{i+1}} C^{i}\xleftarrow{\partial^{i}} C^{i-1}\xleftarrow{}\cdots$$
where $C^i=\text{Hom}(C_i, A)$ and $\partial^i=\text{Hom}(\partial_i,A)$. This is called singular cochain complex. Then similarly we define $H^i(X,A):=\text{ker }\partial^{i+1}/\text{im }\partial^{i}$ (note that indexes are reversed). The fact that arrows are reversed during the construction of cochain complex translates to functoriality of $H^i$: it becomes contravariant, meaning a continuous function $f:X\to Y$ is mapped to $H^i(f,A):=H^i(Y,A)\to H^i(X,B)$ homomorphism (note that $X$ and $Y$ switched places).
That's where co comes from. It has more to do with the fact that cohomology is contravariant, rather then any duality relationship.

Note that there's a variant of homology called "homology with coefficients". We construct it by adding one more step. Similarly to cohomology, we take an abelian group $A$ but instead of applying $\text{Hom}(\cdot, A)$ we apply the tensor product $\cdot\otimes A$ to the chain complex. The tensor product is covariant, so no arrow gets inverted, and the resulting homology is denoted by $H_i(X,A)$. For $A=\mathbb{Z}$ it coincides with the usual definition.
I mention this because you asked in what sense homology is dual to cohomology? Well, it is not. They are however closely related by the so called universal coefficient theorem. In the specific case when $k$ is a field, this theorem implies that
$$H^n(X,k)\simeq \text{Hom}_k(H_n(X,k), k)$$
(because $\text{Ext}^1$ vanishes over fields) which means that cohomology is the dual space of homology. However for non-fields this is no longer true, and the universal coefficient theorem precisely states how they are related (in a somewhat ugly way).
All in all: generally cohomology is not dual to homology in any reasonable way. However they are closely related.
