Definition of a Domain in Pan and Yang's article: “A survey of transfer learning”

Question

In the article "A survey of transfer learning": https://ieeexplore.ieee.org/abstract/document/5288526 Pan and Yang give a good mathematical definition of transfer learning which seems to be widely adopted, given that the article has been cited more than 5000 times.

There is one thing though that confuses me, and that is the definition of a domain and the purpose of the marginal probability distribution. Here is an excerpt from the article:

"In this survey, a domain 𝓓 consists of two components: a feature space 𝓧 and a marginal probability distribution P(X), where $X=\{x_1,…,x_n\}$∈𝓧. For example, if our learning task is document classification, and each term is taken as a binary feature, then 𝓧 is the space of all term vectors, $x_i$ is the ith term vector corresponding to some documents, and X is a particular learning sample. In general, if two domains are different, then they may have different feature spaces or different marginal probability distributions."

Later in the article they say that when two domains are not equal: 𝓓$_S$ ≠ 𝓓$_T$, corresponding to source and target respectively, then that can mean either (or both) of two things:

(1) 𝓧$_S$ ≠ 𝓧$_T$

or

(2) P$_S$(X) ≠ P$_T$(X).

Their example is document classification, where the case of (1) could mean that the two domains are in different languages. In the case of (2) that could mean that the source domain and target domain documents focus on different topics.

What is confusing me here is, how does P(X) represent that? What is the purpose of this P? They call it a marginal probability distribution. Marginal with respect to what, are we summing over some other variables as one does when working with marginal distributions?

I can relate to this f.x. when doing image classification, when one domain is a set of images of animals, while another domain could be a set of peoples faces. But how does the P(X) encode that information? Does P give the probability of an instance x of a random variable X to be included in the domain?

Thanks in advance for all answers :)

Answer 1

Frimann wrote, "Does P give the probability of an instance x of a random variable X to be included in the domain?"

The answer is yes. More specifically, for every possible classifier input $x\in\mathscr{X}$, the "marginal probability distribution" $P(X=x)$ is the probability that $x$ would be randomly sampled from a distribution over $\mathscr{X}$. They are calling it the marginal probability distribution because you may have to sum over other random variables to compute $P(X=x)$.

If the source "domain" is 1024x768 RGB pictures of animals and the target "domain" is 1024x768 RGB pictures of cats, then the space of inputs $\mathscr{X}=\{0,1,2,\ldots, 255\}^{1024\times768\times3}$ is the same for both domains, but the marginal distribution functions are different. The marginal probability function $P_S$ (respectively $P_T$) maps $\mathscr{X}$ into [0,1] and is defined as the probability $P_S(x)$ (respectively $P_T(x)$) that a particular image $x\in\mathscr X$ would be randomly chosen from the set of possible 1024x768 animal images (respectively cat images).