Notation for function where first argument determines domain of second argument $\newcommand{\cM}{\mathcal{M}}
\newcommand{\R}{\mathbb{R}}$I'm writing about a machine learning model selection problem where we have
a finite set $\cM$ of models, each having its own abstract parameter space
$\Theta_M$.
The learning problem induces a reward function $R_M(\theta) : \Theta_M \mapsto \R$
defining the "goodness" of a particular model and choice of parameters for that model.
Although each $R_M$ has a different domain,
semantically they all compute the "goodness" for the same learning task.
In some parts of the exposition, it would help to have a unifying notation
for the "function" $R$ that computes:


*

*inputs: $M \in \cM,\ \theta \in \Theta_M$

*outputs: $R_M(\theta)$.


But it doesn't make sense to define
$R : \cM \times \bigcup_M \{ \Theta_M \} \mapsto \R$
because $R(M_i, \theta \in \Theta_j)$
is undefined when $i \neq j$.
Another place where difficulty arises is talking about the
"universal optimization algorithm" $A$
that, given $M \in \cM$, outputs some $\theta \in \Theta_M$.
Is there a preferred way to handle $R$ and $A$ in notation?
Or, an alternate way to frame the problem that avoids this issue entirely?
 A: The domain of $R$ is an example of dependent sum, also called indexed disjoint union.
When $A$ is a set and $B_a$ is a set for any $a \in A$, the dependent sum is defined as follows:
$$\sum_{a \in A} B_a = \left \{ (a, b) \in A \times \bigcup_{a \in A} B_a : b \in B_a \right \}$$
In your case, you can write:
$$R \colon \sum_{M \in \mathcal M} \Theta_M \to \mathbb R$$
Then $R(M, \theta)$ is defined precisely when $M \in \mathcal M$ and $\theta \in \Theta_M$.
Be aware that this notation, which is common in type theory and category theory, might not be known to your readers. So I would suggest that you define it clearly before you use it.
A: For the domain of $R$ you could write
$$
\{(M,\theta): M\in\mathcal M,\ \theta\in \Theta_M \}.
$$
For $A$ I think you need to declare something like
$$A: M \to \bigcup_M \Theta_M.
$$
You may object that a given $M$ is not “legal” to map to arbitrary elements of 
$\bigcup_M \Theta_M,$
but the fact is that each $M$ actually maps to just one element of
$\bigcup_M \Theta_M,$
and in each case that element happens to be in $\Theta_M.$
You can state that additional restriction after giving the type of $A.$
I think this is an appropriate codomain for $A$ because if you actually implement $A$ as a function in a computer program, you will need it to return something that has “room” for everything in every possible $\Theta_M.$
One way to do that is by having the function return an “Any” type, which would actually be an even larger codomain than
$\bigcup_M \Theta_M.$
