# Minimizing type II error for a test.

Let's say I have a sample of $$n$$ measurements with a sample mean $$\bar{x}$$ and sample variance $$\hat{S}$$. I assume that the distribution is normal with known mean $$\mu$$ and an unknown variance $$\sigma^2$$. I basically want to know how likely is my result $$\bar{x}$$ given the other variables except for the variance.

This is an adaptation of a problem I have where I'm doing an experiment where I want my mean to be $$\mu = 248$$ but I got $$\bar{x} = 249$$ and $$\hat{S} = 2.8$$ with $$n = 25$$ measurements. I'd like to know if this is expected of my experiment or if it's a weird thing such that I'd start doubting if $$\mu$$ really is 248 as I assume. I don't know a lot of statistics so I'm not quite sure how to start solving this puzzle, I tried with confidence intervals and a test that we learned in class where we choose a type I error but those didn't work for me or at least I didn't know how to do it correctly.

• In principle I don't know if the distribution is actually normal, so if there's a way of solving this without knowing the distribution or maybe using the central limit theorem to approximate by a normal then that'd be better. If not then assuming that the distribution is normal is fine I guess. Jun 15, 2019 at 1:10

I believe that using the The Central Limit Theorem and conducting some Hypothesis Tests can help you out.

Recall that the CLT states that if $$x_{1},...,x_{n}$$ is an independent and identically distributed sample coming from some distribution where $$E[x]=\mu$$ and $$Var[x]=\sigma^2<\infty$$ then we can say that $$\frac{\sqrt{n}(\bar{x}-\mu)}{\sigma}$$ converges (in distribution) to a standard normal $$N(0,1)$$.

Now you may want to read up on hypothesis testing, but we can use confidence intervals (C.I.) to try to tackle your question as it is a great starting point for what I believe you are asking ($$H_{0}: \mu = 248$$ versus $$H_{1}: \mu \not = 248$$, note: don't worry if you don't understand this lingo quite yet!).

The formula for a $$100(1-\alpha)\%$$ C.I. when something is distributed as a normal random variable is given by:

$$(\bar{x}-z_{\alpha /2}\frac{s}{\sqrt{n}}, \bar{x}+z_{\alpha /2}\frac{s}{\sqrt{n}})$$

Where $$\bar{x}$$ and $$s$$ are your sample mean and standard deviation respectively. $$n$$ is your number of samples. Finally, $$z_{\alpha /2}$$ is a variable called the critical value and changes depending on a parameter called the type 1 error, $$\alpha$$. Some common values for you to observe:

$$z_{0.05} = 1.645$$, $$z_{0.025} = 1.96$$, and $$z_{0.005} = 2.58$$

If you want to define a $$90\%$$ C.I. for example, we would set $$\alpha =0.10$$ and use $$z_{.10 / 2}=z_{.05}=1.645$$ in interval formula above. Roughly speaking, a $$90\%$$ C.I. can be interpreted as you are $$90\%$$ sure that the true value for $$\mu$$ lies within this interval.

Now because your data might not be coming from a normal distribution, we can only state that the above C.I. is approximately correct. That is, it becomes "more correct" as our sample size, $$n$$, grows.

SOLVING YOUR PROBLEM... 1) Calculate $$\bar{x}, s, n$$ and also choose a value for $$\alpha$$ (commonly one chooses $$\alpha = .01, .05,$$or $$.10$$). Then choose the appropriate value for $$z_{\alpha /2}$$. 2) Calculate the C.I. 3) If your proposed value for what you think $$\mu$$ is (this is called a Null Hypothesis) lies within this interval, then you have enough evidence to not doubt that claim. If your proposed value is outside of the computed C.I. then you can say that your Null Hypothesis is probably not true!

I hope this has given you a little to chew on, and can start a more in depth conversation. Best of luck!

• Yes, we learned about that type of hypothesis test las t class but in this case I believe that doesnt work for me. The problem is that in this case $\alpha$ is the probability that if $\mu = 248$ I reject the hypothesis and so its cool to set it to be a low number, but what I really want is to reduce the probability that I accept (or not reject) the null hypothesis if it is false. That is, I want to minimize the probability of mistakenly thinking $\mu = 248$ Jun 15, 2019 at 5:49
• @Kirtpole: If all you care about is Type II errors (failing to reject the null hypothesis when it is false) but you do not care about Type I errors (rejecting the null hypothesis when it is true) then these is a simple strategy: always reject the null hypothesis no matter whatever evidence you have. Jun 17, 2019 at 0:11
• That's the worst test there is. I could also minimize the Type I error by always accepting the null hypothesis. That's not statistics... Jun 17, 2019 at 4:06