2
$\begingroup$

After learning probability for so many years, I still have trouble with the notation whenever I encounter a new reference. I have pinpointed my confusion to this "two-culture" of probability: one is the probability done by often-times engineers and people who write introductory textbooks on probability (e.g., Sheldon Ross), the other is done by statisticians and more recently practitioners of machine learning.


For the former (engineers and textbook writers), random variables are denoted as $X$, pdf are written as $f_X(x)$, jointed pdf are written as $f_{X,Y}(x,y)$, where $x,y$ are in the range of random variables $X$ and $Y$.

Example: Sheldon Ross, Papolis and Pillai, Leon-Garcia, Bertsekas and Tsitsiklis, Feller, Kobayashi

I found some notes online to illustrate this first culture of probability


For statistics and machine learning, random variables are not denoted as anything, they are suppressed, pdf is written as $p(x)$, where $x$ is the realization of this underlying unspecified random variable, joint pdf are written as $p(x,y)$. Capital letters are almost never used, always lowercase. However, sometimes lower case letter is used to denote a random variable, e.g., $x \sim \mathcal{N}(\mu, \Sigma)$

Example: Bishop, David MacKay, Hastie, Mohri

I found some online notes to illustrate this second culture of probability


Am I correct in my assessment? Can someone who is familiar with this "two-culture" of probability provide a possible reconciliation between these notations?

$\endgroup$
  • 1
    $\begingroup$ This used to bother me alot when I started learning machine learning. Eventually, you just get used to it. $\endgroup$ – Foobaz John Mar 14 at 22:57
  • $\begingroup$ What's worse is when they use $x$ to mean the RV too. For instance, I often see entropy as $H[x]$. $\endgroup$ – user3658307 Mar 16 at 17:09

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.