Proof of the existence of the latent space in deep learning I'm working and studying on generative models. I realized that a strict and mathematical background on the latent space is required. Is there any proof of the existence of the latent space for a manifold? Below is what I've understood about the manifold hypothesis and the latent space:

Manifold hypothesis says; the real world data lie on the lower-dimensional manifold embedded in the original high-dimensional space.

and

A latent space is the lower-dimensional representation of the manifold.

That is, the manifold itself is the lower-dimensional object but embedded (or represented) in the high dimension. For example, consider a high-dimensional space $\mathcal{X}\subset\mathbb{R}^N$ and a manifold $\mathcal{M}\subset\mathcal{X}$. Then, there exist the lower-dimensional representation of the manifold $\mathcal{Z}\subset\mathbb{R}^d$, the corresponding non-linear transformation $T:\mathcal{Z}\to\mathcal{M}$, and the inverse transformation $T^{-1}:\mathcal{M}\to\mathcal{Z}$, where $d \ll N$. In the generative model, the generator network $G$ plays a role of the transformation $T$.
However, is it (mathematically) true (proved)? How can I prove the existence of the latent space?
 A: One way to interpret this is: It is your generative model or transformation $T$ that create a manifold $\mathcal P$. If the generated examples covers the target dataset exactly (all the generated samples are good), then this manifold coincide exactly with our desired data manifold $\mathcal M$, $\mathcal P=\mathcal M$, and provides a parametrization of $\mathcal M$.

As for proof, I think the equivalent condition of existance of latent space is that the "data manifold" is homeomorphic to $\mathbb R^d$  globally (invertible continuous map exist), which is an empirical question.
I'm not sure about existence proofs of the latent space, but there are proofs of non-existance. For example,

*

*if the data manifold $\mathcal M$ is of different dimensions from your latent space $\mathbb R^d$, then there cannot be homeomorphism between them.

*if the data manifold $\mathcal M$ is of non-trivial topology, for example it has the topology of a sphere $S^d$, then it cannot homeomorphic to latent space $\mathbb R^d$.

*if the data "manifold" is not a manifold (in mathematical sense) at all, there cannot be homeomorphic to $\mathbb R^d$. This is very possible. Manifold requires the local dimensionality of anywhere on data manifold is $d$, which may not be true.

So in some sense, I may not take the "manifold" in manifold hypothesis as strictly as in differential geometry....

BTW, I'm also really interested in the geometric interpretation of generative models and I've worked on these ideas a bit!
The Geometry of Deep Generative Image Models and its Applications
