After learning probability for so many years, I still have trouble with the notation whenever I encounter a new reference. I have pinpointed my confusion to this "two-culture" of probability: one is the probability done by often-times engineers and people who write introductory textbooks on probability (e.g., Sheldon Ross), the other is done by statisticians and more recently practitioners of machine learning.

For the former (engineers and textbook writers), random variables are denoted as $X$, pdf are written as $f_X(x)$, jointed pdf are written as $f_{X,Y}(x,y)$, where $x,y$ are in the range of random variables $X$ and $Y$.

Example: Sheldon Ross, Papolis and Pillai, Leon-Garcia, Bertsekas and Tsitsiklis, Feller, Kobayashi

I found some notes online to illustrate this first culture of probability

For statistics and machine learning, random variables are not denoted as anything, they are suppressed, pdf is written as $p(x)$, where $x$ is the realization of this underlying unspecified random variable, joint pdf are written as $p(x,y)$. Capital letters are almost never used, always lowercase. However, sometimes lower case letter is used to denote a random variable, e.g., $x \sim \mathcal{N}(\mu, \Sigma)$

Example: Bishop, David MacKay, Hastie, Mohri

I found some online notes to illustrate this second culture of probability

Am I correct in my assessment? Can someone who is familiar with this "two-culture" of probability provide a possible reconciliation between these notations?

  • 1
    $\begingroup$ This used to bother me alot when I started learning machine learning. Eventually, you just get used to it. $\endgroup$ – Foobaz John Mar 14 at 22:57
  • $\begingroup$ What's worse is when they use $x$ to mean the RV too. For instance, I often see entropy as $H[x]$. $\endgroup$ – user3658307 Mar 16 at 17:09

Mathematicians are stickers for full formal notation.

Statisticians and physicists get a little lazy and use abbreviations. Their shorthand convenience does save space and as long it's clear what they mean it's...okay.

Just be careful.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.