# Bayes' Rule for Parameter Estimation - Parameters are Random Variables?

Let $(\Omega, \mathcal{F}, P)$ be a probability space and let $\mathbf{X}: \Omega \to \mathbb{R}^n$, $\mathbf{Y}: \Omega \to \mathbb{R}^m$ be jointly continuous random vectors. That is, there exists a $\mathcal{B}(\mathbb{R}^n \times \mathbb{R}^m)$-measurable function $f: \mathbb{R}^n \times \mathbb{R}^m \to \mathbb{R}$ such that for all $A \in \mathcal{B}(\mathbb{R}^n)$ and $B \in \mathcal{B}(\mathbb{R}^m)$, $$P((\mathbf{X}, \mathbf{Y}) \in A \times B) = \int_A \int_B f(\mathbf{x}, \mathbf{y}) d\mathbf{y} d\mathbf{x}.$$

Let the marginal densities be denoted by $g(\mathbf{x})$ and $h(\mathbf{y})$ for $\mathbf{x} \in \mathbb{R}^n$ and $\mathbf{y} \in \mathbb{R}^m$, respectively. Then the conditional densities are given by $$g(\mathbf{x}|\mathbf{y}) = \frac{f(\mathbf{x}, \mathbf{y})}{h(\mathbf{y})}, \qquad h(\mathbf{y}|\mathbf{x}) = \frac{f(\mathbf{x}, \mathbf{y})}{g(\mathbf{x})},$$ from which Bayes' theorem follows, $$g(\mathbf{x}|\mathbf{y}) = \frac{h(\mathbf{y}|\mathbf{x}) g(\mathbf{x})}{h(\mathbf{y})}.$$

Here is where I'm a bit confused. I've seen Bayes' theorem applied to estimate the parameters of a probability model, in which we specify the "prior" density of the parameters and then maximize the numerator to obtain a "maximum posterior estimate" of the parameters. Specifically, let $\theta \in \mathbb{R}^n$ denote the model parameters and $\mathbf{y} \in \mathbb{R}^m$ denote the observed data. Then this maximum posterior estimate $\hat{\theta}$ would be given by $$\hat{\theta} = \arg \max_\theta h(\mathbf{y}|\theta) g(\theta).$$

With the above derivation of Bayes' theorem in mind, it seems clear that in order for this to make sense, both the parameters $\theta$ and the random variables $\mathbf{Y}$ must be defined as functions from the sample space to their respective codomains. This holds by definition for $\mathbf{Y}$, but it seems really strange to think of parameters as random variables. But, is this the correct viewpoint for Bayesians? Is there a more clear way to think about this?