What does conditioning on an observed sample mean? Suppose we have a collection of samples $\mathcal{D} = \{x_1, \ldots, x_n\}$ drawn independently from a fixed but unknown distribution $p(x)$, Bayesian estimation uses $\mathcal{D}$ to determine $p(x|\mathcal{D})$.
I am having trouble transforming the above to rigorous mathematical language. The formal definition for conditional distribution is $f|_{X|Y}(x|y) = \frac{f(x,y)}{f(y)}$, but that requires both $X$ and $Y$ to be random variables. I do not know how to think of $\mathcal{D}$ as a random variable.
Should it be thought of as $P(X = x| (X_1, \ldots, X_n) = (x_1, \ldots, x_n))$ where $X_1, \ldots, X_n$ are i.i.d randome variables.
 A: One way to formulate the problem is when you have $k$ different hypotheses $H_1,…,H_k$ with a-priori probabilities $P[H_i]$ (that sum to $1$). Suppose under hypothesis $H_i$ the random vector $\vec{X}$ has joint density $f_{\vec{X}|H_i}(\vec{x})$. You observe a particular realization $\vec{X}=\vec{x}$ and you want to find the most likely hypothesis given this information. So you want to compare for each $i$ the value of:
$$P[H_i|\vec{X}=\vec{x}]$$
But
$$P[H_i|\vec{X}=\vec{x}]=\frac{f_{\vec{X}|H_i}(\vec{x})P[H_i]}{f_{\vec{X}}(\vec{x})} \quad (Eq. *)$$
Since the denominators are the same for all $i$, we just compare numerators and pick the one that is largest. This is called max a-posteriori (MAP) detection.
If you don’t know the a-priori probabilities $P[H_i]$ then you cannot compute the numerators. In that case it is reasonable to pretend the a-priori probabilities are all the same, so you just compare the conditional densities. This is called max likelihood (ML).
Countably infinite case:
If we have hypotheses $\{H_i\}_{i=1}^{\infty}$ the same equation (Eq. *) holds and we again pick the $i \in \{1, 2, 3, ...\}$ with largest numerator.  If $P[H_i]$ are unknown, it is impossible to have $P[H_i]$ the same value for all $i \in \{1, 2, 3, ...\}$, but we can still just "pretend" they are the same and implement the ML rule of choosing the $i$ with largest $f_{\vec{X}|H_i}(\vec{x})$.
Uncountably infinite case
Suppose the hypothesis is defined by random variable $Y$ with uncountably many possibilities.  In this case we must give up trying to guess $Y$ exactly to infinite precision, since we cannot do it.  We can fix a small interval size $\delta>0$ and find the value $y$ for which $Y \in [y, y+\delta]$ is most likely, given our observations:
\begin{align*}
P[Y \in [y, y+\delta]|\vec{X}=\vec{x}] &=\int_{y}^{y+\delta} f_{Y|\vec{X}=\vec{x}}(t)dt\\
&\approx \delta f_{Y|\vec{X}}(y) \\
&= \delta\frac{f_{\vec{X}|Y=y}(\vec{x}|y)f_Y(y)}{f_{\vec{X}}(\vec{x})}
\end{align*}
where the approximation step is accurate when $\delta$ is small. Again the denominators are the same for all options $y \in \mathbb{R}$ so the MAP rule is to pick the $y \in \mathbb{R}$ with the largest numerator $f_{\vec{X}|Y=y}(\vec{x}|y)f_Y(y)$.  If the density $f_Y(y)$ is unknonwn we can implement the ML rule of picking the $y$ with largest conditional density $f_{\vec{X}|Y=y}(\vec{x}|y)$.
