4
$\begingroup$

Suppose that $X$ is a Normal random variable such that:

$$ X \sim N(\mu, \sigma^2) $$

Then, it is common to write:

$$ p(x|\mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(\frac{-(x-\mu)^2}{2\sigma^2}\right) $$

Here, it seems that we are using a conditional notation, but we are conditioning on parameters.

Now suppose that: $Y \sim N(\mu_y, \sigma_y^2)$. If we wrote:

$$ p(X|Y,\mu,\sigma^2) $$

Would this still be valid? I have seen some books combine parameters and random variables in the conditional, but am not sure if this was an abuse of notation or something valid. What is the correct consensus here? Thanks.

$\endgroup$
  • $\begingroup$ I think as long as it's understood that in this case $Y$ to the right of the pipe is shorthand for $Y(\omega)$ for some specific (but possibly unknown) $\omega$, then it's ok. $\endgroup$ – Thoth Jun 24 '17 at 9:41
2
+25
$\begingroup$

I think your question is strongly related to your other question How to show that $f(x,y,\mu\mid a,b,\Sigma) = f(x,y\mid \mu, a,b,\Sigma)\cdot p(\mu)$ if $(x,y)$ are MVN and $a,b,\Sigma$ are hyperparameters?.

I personally don't like to write $p(x | \mu, \sigma^2)$, because I find that confusing with the conditioning notation. I prefer writing $p(x ; \mu, \sigma^2)$ (as I was advised by one of my teachers, and as some authors do). However I am not aware of any convention on the subject, and many authors write $p(x | \mu, \sigma^2)$, consequently mixing parameters and random variables. I consider it as an abuse of notations because this mixes two different things but it remains valid as long as you are aware of what you are writing and if you do not try to apply Bayes rule on fixed parameters. Besides, it is a notation and therefore each author can define it his own way.

$\endgroup$
  • 1
    $\begingroup$ One reason of that is because those author have Bayesian modelling background, or in that context they are actually thinking of a Bayesian model, so those parameters are hyperparameters which is treated as random variable. I agree that it does not like a frequentist setting. And in frequentist settings where those parameters are constant, and thus this notation means you are conditional on a trivial sigma-algebra so it is just the same as the unconditional one. So I guess that's why those author are so free to use (or "abuse" if you like) this notation as it work in both settings. $\endgroup$ – BGM Jul 4 '17 at 5:30
1
$\begingroup$

It is totally proper notation as long as you are using Bayesian approach where $\mu$ and $\sigma^2$ are not regarded as (constant) parameters, albeit a random variables ("prior hyper-parameters"). Assume that $ X \sim N(\mu, \sigma^2)$, and $\mu \sim N(\mu_0, \sigma^2_0$), then by applying the Bayes rule on $P(X|\mu)$ you can get the "posterior hyper-parameters" of $X$. You can use this logic iteratively, and set $\mu_0$ and $\sigma^2_0$ as random variables as well. You can see here for more details

Freqeuntists will use $f_X(x; \mu, \sigma^2)$, when you consider the density of $x$ given (constant) parameters $\mu$ and $\sigma^2$. Although, you may encounter $f(\mu, \sigma^2 | x)$ when you consider the density as a likelihood function (for a sample of one r.v. $X$), however it is not considered as a density function as it defined over the parametric space and not need to satisfy any conditions of proper density\probability function.

$\endgroup$
1
$\begingroup$

If $\mu, \sigma^2$ are actually parameters, then I find $f(x|\mu, \sigma^2)$ a notational abuse and the correct notation would be $$ f(x; \mu, \sigma^2) $$ However, not all authors follow this convention (especially those from Bayesian background) and I have also seen mixing random variables and parameters after the $|$ symbol, which is indeed confusing. Personally, I avoid putting parameters after a conditioning.

From a purely mathematical point of view, however, parameters can be seen as constant random variables generating the trivial $\sigma$-algebra, so conditioning is not technically wrong (although it does nothing). Yet, Bayes' rule does not work if you use that notation and, to me, this is reason enough to avoid it entirely.

On the other hand, if $\mu, \sigma^2$ are (non-constant) random variables, e.g. prior hyper-parameters, then conditioning makes sense.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.