Geometric interpretation of the fundamental theorem for coalgebras? Given an element $m$ in a coalgebra $C$, there always exists a finite-dimensional subcoalgebra $D \subset C$ containing $m$; this is the fundamental theorem for coalgebras. This obviously isn't the case for algebras. However, to make a proper analogue of this theorem for algebras, one should formulate this theorem using arrows and then dualize. An element of $C$ determines a linear map $F:\mathbb{k} \to C$ and vice versa. The fundamental theorem then becomes:
For any linear map $F: \mathbb{k} \to C$ there exists a map $G:\mathbb{k} \to D$, such that $i \circ G=F$, for $D$ some finite dimensional coalgebra, and $i$ an injective map of coalgebras.
Dualizing to algebras, one easily sees that the statement becomes equivalent to the question: is any linear map $f:A \to \mathbb{k}$ zero on an ideal of finite codimension, where $A$ is any algebra. Spelling this out for $k[x]$, one is led to the question: do there exist sequences $(f_{n})_{n}$ that do not satisfy any recursion equation. Does anyone know about a sequence like this, or an existence proof? 
A perhaps less sensical question: does the reformulation of the fundamental theorem in terms of algebras have some nice geometrical interpretation? Say in the case of affine algebras?
 A: Very interesting. Unfortunately the dual statement you suggest is false. Since $k$ is a field, $k [x]$ is a principal ideal domain. In particular, any non-trivial quotient of $k [x]$ is automatically finite-dimensional. Now, any linear map $f : k [x] \to k$ is freely and uniquely determined by the sequence $f(1), f(x), f(x^2), \ldots$, and $f$ factors through the quotient $k [x] / (p(x))$ only if 
$$a_0 f(x^n) + a_1 f(x^{n+1}) + \cdots + a_d f(x^{n+d}) = 0$$
for all $n$, where
$$p(x) = a_0 + a_1 x + \cdots + a_d x^d$$ 
so to find an $f$ that does not factor through any non-trivial quotient of $k [x]$, it is enough to find a sequence that does not satisfy any linear recurrence equation (with constant coefficients). But a sequence that satisfies a linear recurrence equation grows at worst exponentially, so we can just take a super-exponential sequence, say $n!$.
A: Perhaps the following observation is of use. The fundamental theorem of coalgebras can be stated as establishing an equivalence of categories between $Coalg$ and the category $Ind-coalg$. (I will use lower case $c$ in the notation to indicate finite dimensional ones. Here $Ind-coalg$ is the category of 'ind-objects' in $coalg$.) The duality you mention is between $coalg$ and $alg$, so we expect (and obtain) a dualtiy between $Coalg$ and $Pro-alg$, the category of `pro-objects' in $alg$. Under reasonable assumptions on the ground ring, these correspond to pseudo-compact algebras. This leads to two further points.
(i) When dealing with duals in this context, you need to have some continuity assumptions around to get the dual conditions for the coalgebra case.
(ii) For the geometric interpretation note that Grothendieck developed the theory of formal groups using pseudocompact rings and also that for any profinite group, (e.g. Galois groups and algebraic analogues of the fundamental group of a space), the natural analogue of the  group algebra is a pseudocompact ring. 
A: EDIT: In this answer all coalgebras are cocommutative and all algebras commutative
As was already mentioned the statement you mention is equivalent to the statement that there's an equivalence $CoAlg \cong Ind(CoAlg_{fin})$ (everything is over a fixed field $k$). 
We can make this statement more geometric by noticing that 
$$Ind(Coalg_{fin}(k)) \cong Flat(CoAlg_{fin}^{op},Set) \cong Flat(Alg_{fin},Set) \cong Flat(Aff_{fin}^{op},Set)$$ 
Where "$Flat(-,-)$" means flat functors i.e. functors preserving finite limits. This last category is equivalent to the category of pro-representable functors on artin rings or in more geometric terms formal schemes (which are set-theoretically supported on a discrete set of points).
In summary the category of Coalgebras over $k$ is equivalent to the category of Formal Schemes (ind-finite over $k$).
