Square brackets instead of parens for functions? Why are sometimes square brackets used to apply parameters to functions instead of the usual round parentheses?
For instance, in my probability course, they use $\text{P}[X]$ to denote the probability that some event in the set $X$ comes to pass.
$$\text{P}[X] = \sum_{x \in X} p(x)$$
Is there any rule as to when to use square brackets instead of parens or is this arbitrary?
 A: In Jech & Hrbacek's Introduction to Set Theory, the author adopt this notation to avoid confusion about images of sets and images of elements contained in such sets. For instance, is quite common denote $f^{-1}(\{x\}) $ by $f^{-1}(x)$; in the square brackets notation we'd write $f^{-1}[x]$, which is more clean than $f^{-1}(\{x\})$ and not so abusive as $f^{-1}(x)$. Other reason is sets of sets: if we consider a set $A = \{A_1,\dots, A_n\}$ and a function $f:A\to B$ it would not be didactic to write $f(A')$ for some $A'\subseteq A$, for the elements of $A$ is also denoted by capital letters.
A: I can't answer in general, but in probability, square brackets are used to signal notation hacks. It basically means "please don't parse this like usual".
When someone writes $f(something)$, generally what they mean is the following:


*

*$f$ is a function, with well defined domain and codomain (even if these haven't been mentioned explicitly);

*Whatever is written in place of $something$ is a element from the domain, or a variable expression that can only take on values of the domain (that is, it only "evaluates" to values in the domain after the variables are bound)


A probability measure is a function that takes something from the event space (its domain) and output something in $[0,1]$. At first, it seems you could just stick to regular function notation to talk about it. Let's consider a simple bernoulli trial. Its probability measure is:
$P(a) = p\quad $ when $a=1$
$P(a) = (1-p)\quad $ when $a=0$
However, this gets stale fast. Say you have two random variables, $X$ and $Y$. The values $X$ can take are $S_X$, and the values $Y$ can take are $S_Y$. Your sample space is $S= S_X \times X_Y $. How do you write down the probability of $X$ being, say, between 1 and 2, using only regular set theory notation?
Like this:
$P([1,2] \times S_Y)$
or perhaps:
$P( {\langle x,y \rangle : x \in [1,2]})$
The problem here is that we don't care about the values $Y$ takes, but we have to explicitly mention $S_Y$ it in order to construct valid samples. It would be ideal if our notation didn't make us do it (what if we had 10 variables?). So we just use square brackets and write it like this:
$P[ X \in [1,2]]$
$X\in [1,2]$ looks like a boolean value. It, however, should be understood as a subset from the sample space ("all samples in which $X\in [1,2]$"). If you just wrote $P( X \in [1,2])$, with regular parenthesis, this could be parsed as inputting $true$ or $false$ to $P$, which makes no sense.
$P(X∈[1,2]) = P(true) = \ ? \ $ ($true$ is not in the domain of $P$)
You could argue that I'm being too strict: $P( X \in [1,2])$ is perfectly reasonable if you consider $X\in [1,2]$ as a shorthand for an event. But still, you'd have to break the rules to talk about conditional probability:
$P[A|B] \triangleq \dfrac{P[A\cap B]}{P[B]}$
There's just no way to interpret $A|B$ as an event in this case. $P[A|B]$ could be different from every possible value of the original probability measure function! Consider the following probability measure:
$P(\{1\}) = 1/3$
$P(\{2\}) = 1/3$
$P(\{3\}) = 1/3$
In this case, $P(x)$ can evaluate to 0, 1/3, 2/3 or 1. Now let's consider the events $A = \{1\}$ and $B=\{1,2\}$:
$P[A|B] = 1/2$
Since we generally want this more convenient notation, we just use square brackets all the time for probability measures, as a warning that things might get weird.
