Let $(\mathcal X,\mathcal F,\mathcal P)$ be a statistical model with $\mathcal P = \{P_\theta : \theta\in\Theta\}$. A decision rule is a measurable function $\delta:(\mathcal X, \mathcal F)\rightarrow(\mathcal A,\mathcal G)$. Examples for such decision rules include parameter estimates, or hypothesis tests. This question is concerned with certain properties of decision rules.

Definition. A decision rule is said to be unbiased iff $$\mathbb E_{P_\theta}[L(\vartheta, \delta)]\geq\mathbb E_{P_\theta}[L(\theta, \delta)]$$ for each $\theta,\vartheta\in\Theta$.

Assuming a certain loss function $L:(\mathcal A\times\mathcal A, \mathcal G\otimes\mathcal G)\rightarrow(\mathbb R,\mathcal B)$, this yields the familar concepts for unbiasedness of an estimator and unbiasedness of a hypothesis test:

Theorem 1. Assume the $L_2$ loss $L(t,a) = \Vert t - a\Vert_2$, and $\tau:\Theta\rightarrow\mathcal A$. Then $\delta$ is unbiased iff $\mathbb E_{P_\theta}[\delta] = \tau(\theta)$ for all $\theta\in\Theta$, provided that $\mathbb E_{P_\theta}[\Vert\delta\Vert^2_2] < \infty$, and $\mathbb E_{P_\theta}[\delta]\in\tau(\Theta)$.

Theorem 2. Assume the $0$-$1$-loss $L(t,a) = \ell_0 a\mathsf 1_{\Theta_0}(t) + \ell_1(1-a)\mathsf 1_{\Theta_1}(t)$, where $\ell_0,\ell_1>0$, $\Theta = \Theta_1\cup\Theta_0$ with $\Theta_1\cap\Theta_0=\emptyset$, and $\mathsf 1$ is the usual indicator function. Then $\delta$ is unbiased iff $\mathbb E_{P_\theta}[\delta]\leq\alpha$ for all $\theta\in\Theta_0$, and $\mathbb E_{P_\theta}[\delta]\geq \alpha$ for all $\theta\in\Theta_1$, where $\alpha = \frac{\ell_1}{\ell_0+\ell_1}$.

Another important concept in testing and estimation theory is consistency:

  • A sequence of estimtors $\delta_n$ is said to be consistent (for $\tau(\theta)$) iff $P_\theta(\Vert\delta_n - \tau(\theta)\Vert\geq\epsilon)$ converges to zero for each $\theta\in\Theta$.
  • Similarily, a sequence of tests $\delta_n$ is said to be consistent iff $\sup_{n\in\mathbb N}\mathbb E_{P_\theta}[\delta_n]\leq\alpha$ for all $\theta\in\Theta_0$, and $\mathbb E_{P_\theta}[\delta_n]$ converges to $1$ for all $\theta\in\Theta$.

The concepts are clearly related: for estimators, consistency states that the risk converges to zero in probability. For tests it states that the type I error is bounded, while the probability of a type II error converges to zero. It is thus natural to think about generalizations in the decision theory setup stated above. However, I was not able to do so. Is it even possible?

  • 1
    $\begingroup$ Hi! Are you asking how "consistency" can be defined for decision rules? I guess in a way such that the definitions for "estimation" and "tests" are special cases of the general definition? $\endgroup$
    – Idontgetit
    Mar 26, 2021 at 10:50
  • 1
    $\begingroup$ exactly, that's what i want to do $\endgroup$
    – lmaosome
    Mar 26, 2021 at 11:00

1 Answer 1


Definition: We say that a sequence of decisions $\delta_n$ in $\mathcal{A}$ is consistent for the tuple $(L,\tau(\theta),\mathcal{A}',\Theta')$ iff for all $\theta\in\Theta'\subset\Theta$, $\delta_n$ is a sequence in $\mathcal{A}'\subset\{(\mathcal{X},\mathcal{F})→(\mathcal{A},\mathcal{G})\}$ and: $$ L(\tau(\theta),\delta_n)\rightarrow 0,\quad \text{in $P_\theta$-probability} $$ Let's check that we can recover estimation and testing with this definition:

Estimation: Take $L(\tau(\theta),\delta):=||\tau(\theta)-\delta||$, $\Theta' := \Theta$ and $\mathcal{A}':=\mathcal{A}$. Then you recover the given definition of consistency for estimators.

Testing: Here, we take $L(\tau(\theta),\delta):=1-\delta$, $\Theta' := \Theta_1$ $$ \mathcal{A}':=\{\delta:(\mathcal{X},\mathcal{F})→(\mathcal{A},\mathcal{G}):\sup_{n\in\mathbb{N}}\mathbb{E}_{P_\vartheta}[\delta]\leq \alpha,\forall\vartheta\in\Theta_0\} $$ What is left to show is that $1-\delta_n\rightarrow 0$ in $P_\theta$-probability is equivalent to $\mathbb{E}_{P_\theta}[\delta_n]\rightarrow1$, for all $\theta\in\Theta_1$. Using dominated convergence: $$\begin{align*} \lim_{n\rightarrow\infty}\mathbb{E}_{P_\theta}[\delta_n]]&= \lim_{n\rightarrow\infty}\int_0^\infty P_\theta[\delta_n>t]dt\\&= \int_0^\infty \lim_{n\rightarrow\infty}P_\theta[\delta_n>t]dt\\&= \int_0^\infty 1\{t<1\}dt=1 \end{align*}$$

  • $\begingroup$ thanks for your reply! i will try to understand the steps and reward the answer $\endgroup$
    – lmaosome
    Mar 26, 2021 at 14:42
  • 1
    $\begingroup$ can you give a reference? i think there are two minor mistakes: $\mathcal A'$ is a set of decision functions in your setting but $\mathcal A$ is the action space so the former can't be a subset of the latter. Furthermore, $\mathbb E[\delta_n]$ can't depend on the data (only $\delta_n$ does). Finally, I don't understand how the risk can converge to zero in probability if $\theta\in\Theta_0$? $\endgroup$
    – lmaosome
    Mar 27, 2021 at 10:48
  • $\begingroup$ Thanks for the comment, indeed those were two mistakes. I rewrote and simplified my answer a bit. Unfortunately I have no reference. But I think this definition is quite natural. One place where you might find definitions of this flavor is the theory of M-estimation, which aims to be very general. $\endgroup$
    – Idontgetit
    Mar 28, 2021 at 13:43
  • $\begingroup$ "Finally, I don't understand how the risk can convere to zero in probability if $\theta\in \Theta_0$"... For this just modify the definition so that convergence in probability only needs to occur for $\theta$ in some subset of $\Theta$. I wrote it like this, since your definition of consistency for tests asks for $\mathbb{E}_{P_\theta}[\delta_n]\rightarrow 1$ for all $\theta\in\Theta$ (not $\Theta\setminus\Theta_0$). Maybe this was a typo, but anyways I think you get the idea. $\endgroup$
    – Idontgetit
    Mar 28, 2021 at 13:47
  • $\begingroup$ what is still not clear to me is that you define consistency as "$L(\theta, \delta_n)$ converges to zero in probability for all $\theta\in\Theta$". I understand that $L(\theta, \delta_n)$ converges to zero for all $\theta\in\Theta_1$. But how do you verify that $L(\theta, \delta_n)$ converges to zero for $\theta\in\Theta_0$ $\endgroup$
    – lmaosome
    Mar 30, 2021 at 14:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .