Type 1 error condition in one tailed statistical hypothesis test

Consider the following classical statistical test setup:

One assumes a coin to be unfair in the sense that heads, say, occurs more frequently than tails. Thus we set $H_0: p\leq\frac12$ as null hypothesis and $H_1:p>\frac12$ as alternative where $p$ is the probability for heads.

Also let X count the occurence of heads when tossing the coin $n$ times. Given $n$ and a significance level $\alpha$ we get the one-tail condition $$(1)\quad P(X\geq k)\leq\alpha$$ where $P$ has a $(n,p)$-binomial distribution with $p\leq\frac12$ (thus yielding the probability for rejecting $H_0$ when it's actually true).

To solve $(1)$ for $k$ it would now be common (school book) practice to set $p=\frac12$ and solve $(1)$ by inversion. But this isn't correct, as we just know $p\leq\frac12$.

So wouldn't it be better to rather use a distribution for "$k$ wins out of $n$ with a probability of success $\leq\frac12$" and which would that appropriate distribution be?

I want to be more precise: In a more general context the maximum $\alpha$ error could be defined as $$\alpha_{max}:=\max_{\theta\in\Theta_0}\{P_\theta(T(X_1,\dotsc,T_n)\in K)\}$$ where $T$ is some kind of test statistic, in our case counting the number of heads in a sample $X_1,\dotsc,X_n$; $\Theta$ is the parameter space in question (our paramter is $p\sim\theta$), $\Theta_0$ the subspace corresponding to the null hypothesis, i.e. $$H_0: \theta\in\Theta_0,\quad H_1:\theta\in\Theta\setminus\Theta_0;$$ and finally $K$ is the region of rejection of $H_0$, i.e. $$H_0\text{ is rejected iff }T(X_1,\dotsc,T_n)\in K.$$

So in particular we have $\Theta=[0,1], \Theta_0=[0,\frac12]$, yielding $$\alpha_{max}=\max_{p\leq\frac12}\sum_{i=k}^n B_{n,p}(X=i),$$

which should now be $\leq$ a given significance level.

• Jetzt musst du nur noch eine konkrete Frage zu der Ergänzung stellen. Hast du eingentlich schon mal beim Binomialtest in Wiki vorbei geschaut? Commented Aug 4, 2018 at 16:00
• Wieso du hier ein maximales $\alpha$ suchst ist unklar. Beim Hypothesentest ist dieser von vorneherein schon fest gelegt, wie du auch bei meiner Ungleichung in der Antwort siehst. Commented Aug 4, 2018 at 16:14
• Ich verstehe es jetzt so: Das Signifikanzniveau (meist mit $\alpha$ bezeichnet) gibt eine obere Schranke für die W., einen Fehler 1. Art zu begehen, sagen $\alpha_1$. Da man aber unter Annahme von $H_0$ $p$ nicht genau kennt (man hat nur eine Ungleichung), schätzt man $\alpha_1$ durch $\alpha_{max}$ ab. Commented Aug 4, 2018 at 17:46
• Und aus Monotoniegründen ist nun die Summe $\sum B_{n,p}(X=i)$ maximal für $p=p_0=\frac12$ (bei festem $k$) (die W. für mind. $k$ mal Kopf steigt mit der Erfolgwahrscheinlichkeit). Commented Aug 4, 2018 at 17:54

Both Null hypothesis are possible. The crucial point is the definition of the alternative hypothesis, $H_1$. This definition is unique as you can see at the table below. $$\begin{array}{|c|c|c|} \hline &H_0 &H_1 \\ \hline \texttt{two-tailed} & p=p_0 &p\neq p_0 \\ \hline \texttt{right-tailed} & p=p_0 \ \ \text{or } \ \ p\leq p_0 &p>p_0 \\ \hline \texttt{left-tailed} & p=p_0 \ \ \text{or } \ \ p\geq p_0 &p<p_0 \\ \hline \end{array}$$

For the right-tailed case you evaluate the the smallest value of $c$, where

$$\sum_{i=c}^n B(i| p_0,n)\leq \alpha$$

Then the critical range is $\{c, c+1, \ldots, n \}$.

• Alright, but still the question remains why we use $p_0$ in your sum in the cases where $H_0:p\leq p_0$ (or $\geq$): In these cases we do not know the underlying probability distibution to calculate $P(H_0\text{ is true}\land H_0\text{ is rejected})$. Commented Aug 2, 2018 at 18:03
• In your case $p_0=\frac1{2}$ Commented Aug 2, 2018 at 18:06
• Of course, but what if $p=\frac14<\frac12=p_0$. Assuming $H_0$, for all we know this could be the case. Commented Aug 2, 2018 at 18:13
• @DonFuchs We don´t know the real value of $p$-before and after the test. The only statement we can make that is the following. If the estimated value of $p\cdot n$ is in the interval $\{c, c+1, \ldots, n \}$ we do not accept the Null hypothesis with a statistictial significance of $\alpha$. Or we do not reject the alternative hypothesis $p\geq \frac12$ with a statistictial significance of $\alpha$ Commented Aug 2, 2018 at 18:26
• Sorry, but that doesn't convince. The point is: What we do in your sum above is to calculate $P(\text{$H_0$is true}\land\text{$H_0$rejected})$ (which is the type 1 or $\alpha$ error) without really taking the condition $\text{$H_0$is true}$ into account (by additionaly assuming $p$ not to be less than $p_0$). Commented Aug 4, 2018 at 14:08