Minimizing type II error for a test.

Question

Let's say I have a sample of $ n $ measurements with a sample mean $ \bar{x} $ and sample variance $ \hat{S} $. I assume that the distribution is normal with known mean $ \mu $ and an unknown variance $ \sigma^2 $. I basically want to know how likely is my result $ \bar{x} $ given the other variables except for the variance.

This is an adaptation of a problem I have where I'm doing an experiment where I want my mean to be $ \mu = 248 $ but I got $ \bar{x} = 249$ and $ \hat{S} = 2.8 $ with $ n = 25 $ measurements. I'd like to know if this is expected of my experiment or if it's a weird thing such that I'd start doubting if $ \mu $ really is 248 as I assume. I don't know a lot of statistics so I'm not quite sure how to start solving this puzzle, I tried with confidence intervals and a test that we learned in class where we choose a type I error but those didn't work for me or at least I didn't know how to do it correctly.

Answer 1

I believe that using the The Central Limit Theorem and conducting some Hypothesis Tests can help you out.

Recall that the CLT states that if $x_{1},...,x_{n}$ is an independent and identically distributed sample coming from some distribution where $E[x]=\mu$ and $Var[x]=\sigma^2<\infty$ then we can say that $\frac{\sqrt{n}(\bar{x}-\mu)}{\sigma}$ converges (in distribution) to a standard normal $N(0,1)$.

Now you may want to read up on hypothesis testing, but we can use confidence intervals (C.I.) to try to tackle your question as it is a great starting point for what I believe you are asking ($H_{0}: \mu = 248$ versus $H_{1}: \mu \not = 248$, note: don't worry if you don't understand this lingo quite yet!).

The formula for a $100(1-\alpha)\%$ C.I. when something is distributed as a normal random variable is given by:

$(\bar{x}-z_{\alpha /2}\frac{s}{\sqrt{n}}, \bar{x}+z_{\alpha /2}\frac{s}{\sqrt{n}})$

Where $\bar{x}$ and $s$ are your sample mean and standard deviation respectively. $n$ is your number of samples. Finally, $z_{\alpha /2}$ is a variable called the critical value and changes depending on a parameter called the type 1 error, $\alpha$. Some common values for you to observe:

$z_{0.05} = 1.645$, $z_{0.025} = 1.96$, and $z_{0.005} = 2.58$

If you want to define a $90\%$ C.I. for example, we would set $\alpha =0.10$ and use $z_{.10 / 2}=z_{.05}=1.645$ in interval formula above. Roughly speaking, a $90\%$ C.I. can be interpreted as you are $90\%$ sure that the true value for $\mu$ lies within this interval.

Now because your data might not be coming from a normal distribution, we can only state that the above C.I. is approximately correct. That is, it becomes "more correct" as our sample size, $n$, grows.

SOLVING YOUR PROBLEM... 1) Calculate $\bar{x}, s, n$ and also choose a value for $\alpha$ (commonly one chooses $\alpha = .01, .05, $or $.10$). Then choose the appropriate value for $z_{\alpha /2}$. 2) Calculate the C.I. 3) If your proposed value for what you think $\mu$ is (this is called a Null Hypothesis) lies within this interval, then you have enough evidence to not doubt that claim. If your proposed value is outside of the computed C.I. then you can say that your Null Hypothesis is probably not true!

I hope this has given you a little to chew on, and can start a more in depth conversation. Best of luck!

References for you [1] https://support.minitab.com/en-us/minitab-express/1/help-and-how-to/basic-statistics/inference/supporting-topics/basics/what-is-a-hypothesis-test/ [2] http://onlinestatbook.com/2/estimation/mean.html [3] https://en.wikipedia.org/wiki/Central_limit_theorem