# Consistency in Statistical Decision Theory

Let $$(\mathcal X,\mathcal F,\mathcal P)$$ be a statistical model with $$\mathcal P = \{P_\theta : \theta\in\Theta\}$$. A decision rule is a measurable function $$\delta:(\mathcal X, \mathcal F)\rightarrow(\mathcal A,\mathcal G)$$. Examples for such decision rules include parameter estimates, or hypothesis tests. This question is concerned with certain properties of decision rules.

Definition. A decision rule is said to be unbiased iff $$\mathbb E_{P_\theta}[L(\vartheta, \delta)]\geq\mathbb E_{P_\theta}[L(\theta, \delta)]$$ for each $$\theta,\vartheta\in\Theta$$.

Assuming a certain loss function $$L:(\mathcal A\times\mathcal A, \mathcal G\otimes\mathcal G)\rightarrow(\mathbb R,\mathcal B)$$, this yields the familar concepts for unbiasedness of an estimator and unbiasedness of a hypothesis test:

Theorem 1. Assume the $$L_2$$ loss $$L(t,a) = \Vert t - a\Vert_2$$, and $$\tau:\Theta\rightarrow\mathcal A$$. Then $$\delta$$ is unbiased iff $$\mathbb E_{P_\theta}[\delta] = \tau(\theta)$$ for all $$\theta\in\Theta$$, provided that $$\mathbb E_{P_\theta}[\Vert\delta\Vert^2_2] < \infty$$, and $$\mathbb E_{P_\theta}[\delta]\in\tau(\Theta)$$.

Theorem 2. Assume the $$0$$-$$1$$-loss $$L(t,a) = \ell_0 a\mathsf 1_{\Theta_0}(t) + \ell_1(1-a)\mathsf 1_{\Theta_1}(t)$$, where $$\ell_0,\ell_1>0$$, $$\Theta = \Theta_1\cup\Theta_0$$ with $$\Theta_1\cap\Theta_0=\emptyset$$, and $$\mathsf 1$$ is the usual indicator function. Then $$\delta$$ is unbiased iff $$\mathbb E_{P_\theta}[\delta]\leq\alpha$$ for all $$\theta\in\Theta_0$$, and $$\mathbb E_{P_\theta}[\delta]\geq \alpha$$ for all $$\theta\in\Theta_1$$, where $$\alpha = \frac{\ell_1}{\ell_0+\ell_1}$$.

Another important concept in testing and estimation theory is consistency:

• A sequence of estimtors $$\delta_n$$ is said to be consistent (for $$\tau(\theta)$$) iff $$P_\theta(\Vert\delta_n - \tau(\theta)\Vert\geq\epsilon)$$ converges to zero for each $$\theta\in\Theta$$.
• Similarily, a sequence of tests $$\delta_n$$ is said to be consistent iff $$\sup_{n\in\mathbb N}\mathbb E_{P_\theta}[\delta_n]\leq\alpha$$ for all $$\theta\in\Theta_0$$, and $$\mathbb E_{P_\theta}[\delta_n]$$ converges to $$1$$ for all $$\theta\in\Theta$$.

The concepts are clearly related: for estimators, consistency states that the risk converges to zero in probability. For tests it states that the type I error is bounded, while the probability of a type II error converges to zero. It is thus natural to think about generalizations in the decision theory setup stated above. However, I was not able to do so. Is it even possible?

• Hi! Are you asking how "consistency" can be defined for decision rules? I guess in a way such that the definitions for "estimation" and "tests" are special cases of the general definition? Commented Mar 26, 2021 at 10:50
• exactly, that's what i want to do Commented Mar 26, 2021 at 11:00

Definition: We say that a sequence of decisions $$\delta_n$$ in $$\mathcal{A}$$ is consistent for the tuple $$(L,\tau(\theta),\mathcal{A}',\Theta')$$ iff for all $$\theta\in\Theta'\subset\Theta$$, $$\delta_n$$ is a sequence in $$\mathcal{A}'\subset\{(\mathcal{X},\mathcal{F})→(\mathcal{A},\mathcal{G})\}$$ and: $$L(\tau(\theta),\delta_n)\rightarrow 0,\quad \text{in P_\theta-probability}$$ Let's check that we can recover estimation and testing with this definition:
Estimation: Take $$L(\tau(\theta),\delta):=||\tau(\theta)-\delta||$$, $$\Theta' := \Theta$$ and $$\mathcal{A}':=\mathcal{A}$$. Then you recover the given definition of consistency for estimators.
Testing: Here, we take $$L(\tau(\theta),\delta):=1-\delta$$, $$\Theta' := \Theta_1$$ $$\mathcal{A}':=\{\delta:(\mathcal{X},\mathcal{F})→(\mathcal{A},\mathcal{G}):\sup_{n\in\mathbb{N}}\mathbb{E}_{P_\vartheta}[\delta]\leq \alpha,\forall\vartheta\in\Theta_0\}$$ What is left to show is that $$1-\delta_n\rightarrow 0$$ in $$P_\theta$$-probability is equivalent to $$\mathbb{E}_{P_\theta}[\delta_n]\rightarrow1$$, for all $$\theta\in\Theta_1$$. Using dominated convergence: \begin{align*} \lim_{n\rightarrow\infty}\mathbb{E}_{P_\theta}[\delta_n]]&= \lim_{n\rightarrow\infty}\int_0^\infty P_\theta[\delta_n>t]dt\\&= \int_0^\infty \lim_{n\rightarrow\infty}P_\theta[\delta_n>t]dt\\&= \int_0^\infty 1\{t<1\}dt=1 \end{align*}
• can you give a reference? i think there are two minor mistakes: $\mathcal A'$ is a set of decision functions in your setting but $\mathcal A$ is the action space so the former can't be a subset of the latter. Furthermore, $\mathbb E[\delta_n]$ can't depend on the data (only $\delta_n$ does). Finally, I don't understand how the risk can converge to zero in probability if $\theta\in\Theta_0$? Commented Mar 27, 2021 at 10:48
• "Finally, I don't understand how the risk can convere to zero in probability if $\theta\in \Theta_0$"... For this just modify the definition so that convergence in probability only needs to occur for $\theta$ in some subset of $\Theta$. I wrote it like this, since your definition of consistency for tests asks for $\mathbb{E}_{P_\theta}[\delta_n]\rightarrow 1$ for all $\theta\in\Theta$ (not $\Theta\setminus\Theta_0$). Maybe this was a typo, but anyways I think you get the idea. Commented Mar 28, 2021 at 13:47
• what is still not clear to me is that you define consistency as "$L(\theta, \delta_n)$ converges to zero in probability for all $\theta\in\Theta$". I understand that $L(\theta, \delta_n)$ converges to zero for all $\theta\in\Theta_1$. But how do you verify that $L(\theta, \delta_n)$ converges to zero for $\theta\in\Theta_0$ Commented Mar 30, 2021 at 14:44