More on the versions of the Peter-Weyl theorem The following three statements appear analogous: 


*

*For a finite group $G$, the group algebra $\mathbb C[G]$ decomposes as $\bigoplus_{V {\rm\ irred}} V^* \otimes V$. 

*(Peter-Weyl) For a compact group $G$, the space of square-integrable functions $L^2(G)$ is the $L^2$-completion of $\bigoplus_{V {\rm\ unitary\ irred}} V^* \otimes V$. 

*For a complex reductive group $G$, the coordinate algebra ${\mathcal O}[G]$ decomposes as $\bigoplus_{V {\rm\ finite\ irred}} V^* \otimes V$.
The first space is a function algebra under convolution product and the third under pointwise multiplication. (I'm not quite sure now about the second -- does it have an algebra structure?). The decompositions are as representations where $G \times G$ acts on each function space by left/right translation, so the algebra structure doesn't matter. But still -- why this lack of symmetry? Is that really how it is? 
 A: In each of these cases, you have a group $G$ (with various additional structures), and a space of functions on that group, which is therefore naturally a $G \times G$ module under left and right translation.
But "where there's a function space, there's a distribution space". In each case the function space has a natural topology, and you can consider the space of continuous linear maps from the function space to the base field. This doesn't see the pointwise product structure on functions, but it has a new product structure (generally noncommutative) given by convolution; and this new space also has a $G \times G$ action by transport of structure. If $G$ is compact, then both the function space and its dual will (in all the examples I can think of) look like some kind of completion of the direct sum of $V \otimes V^*$ over all finite-dimensional irreps, but the functions and the distributions will generally use different notions of completion.
For $\mathbb{C}[G]$ when $G$ is finite, there is the (commutative) pointwise-product-of-functions structure, but the linear dual of $\mathbb{C}[G]$ is canonically $\mathbb{C}[G]$ itself and this picks up the convolution product structure. And you get the same action of $G \times G$ both ways round. For $L^2$ functions on a compact Lie group you get the same thing happening: the continuous dual of $L^2(G)$ is $L^2(G)$ itself, so you just get the same space, with the same topology. But when $G$ has positive dimension there's no natural pointwise product of two $L^2$ functions, so you miss part of the picture.
For $\mathcal{O}[G]$ where $G$ is a reductive algebraic group, you have a function space and a distribution space, and the functions have a pointwise product and the distributions a convolution product. But the two spaces have very different topologies. The topology on $\mathcal{O}[G]$ is the finest locally convex topology, for which every subspace is closed and every linear functional is continuous. Since this is an insanely strong topology, the dual topology will have to be very weak, and if you chase through definitions for a while, you can convince yourself that the dual of 
$$ \mathcal{O}[G] = \bigoplus_V V \otimes V^*$$
is the direct product
$$ \mathcal{O}[G]^* = \prod_V V \otimes V^*. $$
In general function spaces won't be self-dual -- that's a really special thing about $L^2$ spaces. So the lack of symmetry you mention, that there are two natural objects which are dual to each other but not isomorphic, is the generic case.
A: One should also point out this question, which proves that, for $V$ a simple representation of $G$, the $V$ isotypic component of $\mathbb{C}[G]$ is $V \otimes V^{\vee}$. That question asks in the algebraic context, but the proof is formal nonsense which works in any reasonable setting.
So it is easy to see that the union of the finite dimensional subreps of $\mathbb{C}[G]$ is always $\bigoplus V \otimes V^{\vee}$. The nontrivial thing is to see that this is dense in whatever function/distribution space you are considering (as discussed in David Loeffler's excellent answer).
