11
$\begingroup$

This question is about notation for random variables (RVs), distributions and pdfs/pmfs and their common (ab)use as I recently got confused.

Let $X,Y$ denote random variables.

First, notations I usually encounter. Please correct me:

  • values a RV takes on are usually denoted by small caps so that $P(X=x) \in [0,1]$ denotes the probability of the RV $X$ taking on the value $x$
  • $X_1,...,X_n \sim X$ means "let X_1,...,X_n be RV with same distribution as $X$" (often $\overset{\text{iid}}{\sim}$)
  • if $X$ is discrete it's pmf is usually denoted by $p(x) = p_X(x) = P(X=x) \in [0,1]$
  • if $X$ is non-discrete it's pdf is usually denoted by $f(x) = f_X(x) \in [0,\infty)$ or $p(x) = p_X(x)$ to easily talk about discrete and non-discrete RVs at the same time
  • the cdf is usually written as $F(x) = F_X(x) = P(X \leq x)$ which is a sum/integral using the pdf/pmf

The following notations I've usually understood in an "intuitive" way or assumed to just be sloppy but caused some confusion:

  • "Let $X$ be a RV with distribution $X \sim P(X)$" -- What exactly is meant? Should I think of $P$-robability here or is it a symbol which reads "this denotes/represents the distribution of $X$"?
  • "$p(X,Y), p(X), p(X|Y)$ denote the joint, marginal, conditional probability density functions" -- How should I understand this? I mean, they should be functions of values the RVs can take on but here they take the RVs itself as argument?
  • " Let $P(x,y)$ be an (unknown) joint probability distribution on instances and labels $X × Y$. Given a training sample ${(x_i, y_i)}_{i=1}^n \overset{\text{iid}}{\sim} P(x,y)$ ..." -- How to read this?

Could someone help me out and shed some light upon above mentioned points?

Sorry, if my questions are stupid. I just feel the notation gets far more sloppy when reading applied stuff and it would help me to pin down what actually is meant or to know that one needs to relax and learn how to sloppily-correctly read this.

$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

You are quite right. The second bullet list is full of muddled notation, which shows that the author has vague and confused ideas about probability. Rather than trying to interpret and learn from this sort of stuff, you would be much better off sticking to material by authors whose notation makes sense.

$\endgroup$
7
  • 2
    $\begingroup$ Thanks. I still suspect, that there is some sense in this notation as it is e.g. also done in slides of a course I worked through on my own. But definitely a very uncommon notation. So far I took it as $p_{X,Y} \equiv p(X,Y), p_X \equiv p(X), p_{X|Y} \equiv p(X|Y)$ which would allow intuitive notions like $p(X,Y) = p(X|Y)p(Y)$. Does this make sense? (e.g. gatsby.ucl.ac.uk/teaching/courses/ml1-2008/lect1-handout.pdf slides 24, 31) $\endgroup$
    – JamHei
    Commented Dec 15, 2014 at 13:49
  • 2
    $\begingroup$ Still, these are awfully sloppy and have dreadful consequences as far as the understanding of students is concerned (for tons of examples, see math.se questions...). $\endgroup$
    – Did
    Commented Dec 15, 2014 at 14:47
  • $\begingroup$ @JamHei: Generously enough interpreted, everything makes sense; and your interpretation of "$p(X,Y)$" as meaning $p_{X,Y}$ would make sense here. But having to deal with this sort of muddle is a big handicap for learners. $\endgroup$
    – John Bentin
    Commented Dec 15, 2014 at 14:53
  • $\begingroup$ Thanks. I am not defending this notation for learners. In fact they're the reason for my confusion. I just want to make sense out of it and understand "how to read them" in this context e.g. of this course or other more application oriented articles I've seen them in as well. Like recapping what they actually mean or what sloppy notations are commonly used. What about the other two bullets? $\endgroup$
    – JamHei
    Commented Dec 15, 2014 at 15:37
  • $\begingroup$ PS. On Wikipedia (en.wikipedia.org/wiki/Posterior_probability#Definition) for example they use the term "probability distribution function" (en.wikipedia.org/wiki/Probability_distribution_function) in case of $p(\theta|x)$ which is kind of the same thing (as I take $\theta$ and $x$ to be RVs here). So it might be common to this field? $\endgroup$
    – JamHei
    Commented Dec 15, 2014 at 15:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .