Probability that an observation is from a known population

Question

This seems like it should be an easy question, at least the first part, but I can't think of the right way to do it or the search terms needed to google it.

Given a normal random variable $X$ with known distribution, what is the probability that an observation $r$ came from $X$? Is this even a well-posed question, or do I need to define some priors?

My end goal is, given a large set of random variables, $X_1 \dots X_n$, I want a quick way of assigning probabilities that a given measurement is from each random variable. Assuming I can reasonably define a closeness metric, $\rho_i$, for the measurement to each $X_i$ I can do $$ \begin{align} P(r \text{ is from} X_i) &= \frac{\rho_i}{\sum_{k=1}^n \rho_k} \end{align} $$

Bonus: What if the observation $r$ includes measurement noise (normally distributed)?

Double bonus: How do I extend this to bivariate normal distributions?

Answer 1

If you have a number of random variables with known distribution and you know that a value $r$ is realised by exactly one variable. Then one can define the distribution of $Y$ ($Y=i$ if $r$ from $X_i$) as being:

$$P(Y=i)= \frac{f_i(r)}{\sum_{k=1}^n f_k(r)}$$

where $f_i(x)$ is the density function of $X_i$.

Intuitively working with densities makes sense, because if the $X_i$ are continuous and $f_i(r)> f_j(r)$ for a $i$ and $j$ then there is a $\delta$ such that $P(X_i \in (r-\delta,r+\delta)) > P(X_j \in (r-\delta,r+\delta))$. But densities are not probabilities, so caution is advised.

For the bonus: If you include noise which is normally distributed with mean 0 and variance $\sigma^2$, one can take $(r-\epsilon,r+\epsilon)$ with $\epsilon$ chosen so you have coverage with which you feel comfortable. Now you can use normal probabilities and state:

$$P(Y=i) =\frac{P(X_i \in (r-\epsilon,r+\epsilon))}{\sum_{k=1}^n P(X_ \in (r-\epsilon,r+\epsilon))}$$