0
$\begingroup$

Suppose $X\sim N(\mu,\sigma)$. I want to find the following probability $P[\mu \ge \theta |x= \theta -c]$ for $c>0$.

In another word, I saw a sample of Normal distribution, $x$, and know that it is smaller than $\theta$. Now I want to know what is the probability that the mean of distribution is larger than $\theta$.

Attempt 1 I suppose, $\mu$ is a random variable and now with the observation $x=\theta-c$, I want to find $P[\mu\ge \theta]$

$$P[\mu \ge \theta |x= \theta -c]= \frac{P[x=\theta-c|\mu\ge\theta]\times P[\mu\ge\theta]}{P[x=\theta -c]}$$ $$=\frac{\int_\theta^\infty P[x=\theta-c|\mu=t] \times P[\mu=t] dt}{P[x=\theta -c]}$$

Now the only thing I know is that $P[x\le \theta -c|\mu=t]=\Theta (\theta -c,N(t,1)$. I don't have prior information about $P[\mu\ge\theta]$ and say I can assume any distribution that makes the analysis easy.

Attempt 2 I don't have any information about $P[\mu\ge \theta]$ or $P[x=\theta-c]$, so is it acceptable to assume they are uniform on their space?

If so I can do the following

$$P[\mu \ge \theta |x= \theta -c]=P[x=\theta-c|\mu\ge\theta]\\ =\int_\theta^\infty \phi(\mu=t,\sigma,\theta -c) dt =\int_\theta^\infty\frac{1}{\sigma\sqrt{2\pi}}e^{\frac{(t-\theta+c)^2}{2\sigma}}\\ =\int_\theta^\infty\frac{1}{\sigma\sqrt{2\pi}}e^{\frac{(-t+\theta-c)^2}{2\sigma}} =\int_\theta^\infty \phi(\mu=\theta,\sigma,t+c) dt\\ =\Theta(\mu=\theta,\sigma,\theta+c)$$

which is just CDF of $N(\theta,\sigma)$ for $\theta-c$.

$\endgroup$
5
  • $\begingroup$ What. The mean of the distribution is not a random variable. $\endgroup$ Jun 5, 2014 at 2:56
  • $\begingroup$ May be a little background helps. I have a data-structure that can estimate the number of distinct items. Its error is Normal. Using the output of it that is below a threshold $\theta$, I want to know what is the probability that the real value (mean) was above the threshold $\theta$. $\endgroup$
    – Masood_mj
    Jun 5, 2014 at 3:02
  • $\begingroup$ My intuition says that it should be similar to $N(\mu,\sigma)$ without any prior information because suppose that I assume a random variable with mean $\mu'$ and move it iteratively from $\theta$ to infinite and find the probability that the sample is $\theta -c$. $\endgroup$
    – Masood_mj
    Jun 5, 2014 at 3:07
  • $\begingroup$ Do you know $\sigma^2$? $\endgroup$
    – Henry
    Jun 5, 2014 at 6:38
  • $\begingroup$ Yes. Sorry that I forgot to mention it $\endgroup$
    – Masood_mj
    Jun 5, 2014 at 12:20

2 Answers 2

1
$\begingroup$

You said you can assume any prior on $\mu$ that makes the problem easy. Easiest one to assume then is $\mu \sim N(\alpha, \beta^2)$ where $\alpha,\beta$ are known so $X = \mu + N(0, \sigma^2)$ with the $N(0,\sigma^2)$ independent of $\mu$. Then, $X$ and $\mu$ are jointly gaussian, and the conditional distribution of $\mu$ given $X$ is a Gaussian with mean as the linear MMSE estimate of $\mu$ given $X$, and its variance is the covariance of the error in the linear MMSE estimator.

The linear MMSE estimator of $X$ given $Y$ is $E[X] + cov(X,Y) cov(Y,Y)^{-1} (Y - E[Y])$ covariance of the error given by $cov(X) - cov(X,Y) cov(Y,Y)^{-1} cov(Y,X)$.

In this case, you need $E[X] = E[E[X|\mu]] = E[E[\mu + N(0,\sigma^2)|\mu]] = E[\mu + E[N(0,\sigma^2)|\mu]] = E[\mu + 0] = \alpha$, $cov(\mu, \mu) = \beta^2$, $cov(X) = cov(\mu) + cov(N(0,\sigma^2)) = \beta^2 + \sigma^2$ and $cov(X,\mu) = cov(\mu + N(0,\sigma^2), \mu) = cov(\mu) + cov(N(0,\sigma^2)) = \beta^2 + 0 = \beta^2$ and similarly, $cov(\mu,X) = \beta^2$.

This specifies the distribution of $\mu | X$ in this case, and from that, you can calculate all the desired probabilities.

$\endgroup$
4
  • $\begingroup$ That's a great first step and is valuable in theory. But in practice I don't know $\alpha$ and $\beta$. $\endgroup$
    – Masood_mj
    Jun 7, 2014 at 12:51
  • $\begingroup$ Well, you need a distribution to have a probability. So, pick one, find the conditional distribution and then you get the probability. Normally you're interested in nearly the other way around - $P(X> \mu + c)$ for $c>0$ when $\mu$ is known, which can be treated with Chernoff bounds. $\endgroup$
    – Batman
    Jun 7, 2014 at 14:26
  • $\begingroup$ I see. By the way, for this example you say, why do I need chernoff. Isn't it just 1-CDF of Gaussian distribution for $c$ ? $\endgroup$
    – Masood_mj
    Jun 7, 2014 at 16:10
  • $\begingroup$ You get a reasonably tight bound on the probaiblity for sufficiently large $c$ which is easier to deal for a lot of purposes. You certainly don't need chernoff. $\endgroup$
    – Batman
    Jun 7, 2014 at 17:03
0
$\begingroup$

Well, it is just $1/2$. Because the observation is either above or below the mean of the distribution. You need to specify more information. Also, your problem is ill-defined, because the mean is not a random variable. I assumed that you want to know what is the probability that you observe an $x$ that satisfies the inequality.

$\endgroup$
1
  • $\begingroup$ No Taha, I wan t to know $x=\theta-c$ not $x<\theta$ $\endgroup$
    – Masood_mj
    Jun 5, 2014 at 2:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .