what does "patch" mean on manifold? In this paper, the author introduces a generalization of CNNs on 2-dimensional manifolds$\chi$. And named the patch operator
$$(D(x)f)(\rho,\theta)=\int_\chi w_{\rho,\theta} (x,y)f(y)dy$$
and this maps the values of the function $f$ at a neighborhood od the point $x\in\chi$ into the local polar coordinates $\rho,\theta$.
The point is, what does "patch" actually mean? they say $D(x)f$ can be regarded as a patch on the manifold but I don't understand.
Can someone tell me please? thank you.
 A: Let's go back to the classical convolutional network for a moment. We have an image $I$, which is a Euclidean manifold. We then convolve the kernels $k(u,v)$ over the image $I(x,y)$. If we consider the patch centered at $(x,y)$ on $I$, then it is clear how to do this: just apply $k(i,j)$ to $I(x-i,y-j)$. 
In other words, applying a convolution operation to a patch around $(x,y)$ means simply extracting a square piece of the image with dimensions equal to those of the filter kernel.
But such a patch is not well-defined for non-Euclidean manifolds! 
There are no global coordinates on such a surface, unlike for $I$. 
This is related to the problem of surface parameterization: in the case of 2D manifolds in 3D, it boils down to finding a map (in the context of texturing called a UV map) from a plane to the surface. 
Ideally, one would like to be able to draw a nice 2D square in the 2D parameter space of the surface (for instance), and get a corresponding patch on the 3D shape. Unfortunately, it is not possible to find a perfectly area-preserving map like this in general. (Hence why conformal maps are usually used instead).
So how can we get patch on the surface to convolve our kernel to?
There are several problems: 


*

*On $I$, we can draw a nice square where we know that e.g. the horizontal vs vertical distances are equal (say 3 array cells distance in each direction). But a highly curved patch will cover different areas depending on the local curvatures! (The size of a local area element is determined by the spatially varying Riemannian metric tensor, in other words).

*On $I$, every patch is always in a consistent orientation. In other words, moving in $x$ always moves horizontally, so the network can trust that e.g. a horizontally oriented filter array is always horizontally oriented. But this is not the case on a non-Euclidean surface! In other words, if I have a 2D filter kernel array (as for an image), how do I orient it before applying it to the surface? 

*Each output value (for a given node) is a weighted sum over the nearby patch (i.e., nearby nodes). Of course, for a discretized manifold, we often have features only per node. For a CNN on $I$, we can easily associate kernel values to image features, but how can we do that here? In other words, for each kernel value, which feature on the manifold do I associate it to?
In Masci et al's work, these issues are resolved by 
(a) switching to geodesic polar coordinates,
(b) extracting a feature map patch using a weighted sum over nearby feature nodes,
and
(c) taking the maximum filter response over all possible orientations (this resolves the orientation ambiguity of (2)).
Notice that using polar coordinates $\rho,\theta$ sort of moves the 2D problem of ambiguity of coordinates into 1D, because $\rho$ has no orientation, and hence is always well-defined in terms of the network learning how to use its kernels.
Using geodesic distances in (a) helps resolve problem 1, since it accounts for the manifold curvature, while problem 3 is resolved by (b) using the patch operator, which takes a neighbourhood $N(p)$ around a point $p$ and the feature signal $f(q)$ and maps it to a "patch" that is parameterized in polar coordinates, where the signal at each each coordinate position is computed by integrating over the patch. 
(Recall that for $I$, each kernel value is easily mapped to its corresponding value in the patch extracted from $I$, while on the manifold if we have a kernel value for some $t=(\rho,\theta)$ then what is the corresponding feature value from the signal on the manifold? We may not have a node at exactly that position $t$! So we have to interpolate that value from nearby nodes so that we can multiply it to the kernel associated to $t$. This interpolation is done via the "patch operator").
So, to your question:

The point is, what does "patch" actually mean?

The patch is a "piece of the manifold" that has been parametrized in a way that makes it amenable to apply a convolutional filter to the features of that manifold. Our kernels will be parametrized by $\rho,\theta$ instead of a local Cartesian $x,y$. 
Consider a Riemannian manifold $(\mathcal{X},g)$. Let $f:\mathcal{X}\rightarrow \mathbb{R}$ be a scalar signal on $\mathcal{X}$ and fix $q\in \mathcal{X}$. We want a function $\eta$ that takes in a point $q$ and a scalar signal $f$, and maps it to a function $F(\rho,\theta)$ that maps a local point in polar coordinate to the value of the scalar field at that point, as a weighted sum of the signal at the points nearby.
Call $\eta(q,f) = D(x) f$ and $F(\rho,\theta) = (\eta(q,f))(\rho,\theta) = (D(x) f)(\rho,\theta)$.
Define a weighting function $w_{(\rho,\theta)}(q,p)$, which gives the weighting of the signal stored at position $p\in\mathcal{X}$, for the output signal at position $(\rho,\theta)$, for the patch centered at $q\in\mathcal{X}$. Then the "patch" of feature values is given by
$$
(D(q) f)(\rho,\theta) = \int_{\mathcal{X}} w_{(\rho,\theta)}(q,p) f(p)\,dp.
$$
For instance, if $q$ is far from $p$, then the weighting function is likely to be small. So at a position near $q$, given by $(\rho,\theta)$in the local coordinates centered at $q$, the contribution of the signal at $p$ (written $f(p)$) will be very small.
In the Euclidean case, each "node" is a pixel. 
Since we always have a known value at each point to associate to the kernel, we use a weighting function given by a Dirac Delta functional $\delta$.
We also extract a patch in $(x,y)$ rather than $(\rho,\theta)$ because our Euclidean distance already is the geodesic distance, and there is no need to worry about rotational ambiguity (usually anyway).
So an image $I$ extracts a patch via:
\begin{align}
(D(q)f)(x,y) 
&= \int_I w_{x,y}(q,p) f(p)\,dp \\
&= \int_I \delta_{x,y}(p) f(p)\,dp \\
&= f(x,y) \\
\end{align}
where $\delta_{x,y}(q-p)$ is centered at $(x,y)$ and in local coordinates $q-p = (x,y)$. So on $I$, evaluating the patch at $x,y$ simply gives you the pixel value at that point.
References


*

*Masci et al, Geodesic convolutional neural networks on Riemannian manifolds.

*Monti et al, Geometric deep learning on graphs and manifolds using mixture model CNNs.
