# Square brackets instead of parens for functions?

Why are sometimes square brackets used to apply parameters to functions instead of the usual round parentheses?

For instance, in my probability course, they use $\text{P}[X]$ to denote the probability that some event in the set $X$ comes to pass.

$$\text{P}[X] = \sum_{x \in X} p(x)$$

Is there any rule as to when to use square brackets instead of parens or is this arbitrary?

• Who knows? As far as I can tell, it's arbitrary. I would avoid square brackets in general. – Qiaochu Yuan Feb 26 '13 at 19:14
• I use brackets for stylistic reasons, with no different meaning from parentheses, whenever I judge that too many nested parentheses would look confusing/ugly; for instance I might write $f[g(x)]$ instead of $f(g(x))$. – user7530 Feb 26 '13 at 19:14
• I used to distinguish between $\mathbb{P}[\cdots]$ and $\mathbb{P}(\cdots)$: in the former, the description of the set to be measured goes between the brackets, and in the latter, the set to be measured goes between the parentheses. So, for example, $\mathbb{P}[X = x]$ but $\mathbb{P}(\{ x \})$. – Zhen Lin Feb 26 '13 at 19:16

In Jech & Hrbacek's Introduction to Set Theory, the author adopt this notation to avoid confusion about images of sets and images of elements contained in such sets. For instance, is quite common denote $f^{-1}(\{x\})$ by $f^{-1}(x)$; in the square brackets notation we'd write $f^{-1}[x]$, which is more clean than $f^{-1}(\{x\})$ and not so abusive as $f^{-1}(x)$. Other reason is sets of sets: if we consider a set $A = \{A_1,\dots, A_n\}$ and a function $f:A\to B$ it would not be didactic to write $f(A')$ for some $A'\subseteq A$, for the elements of $A$ is also denoted by capital letters.

• $f^{-1}(\{X\})$ and $f^{-1}[X]$ are different things. $f^{-1}[X]$ is the preimage of a subset $X$ of the codomain and $f^{-1}(x)$ is the preimage of an element of the codomain (only if f is injective). For example, consider a function $f$ from a two-element set $\{a,b\}$ to a two-element set $\{x, \{x\}\}$ such that $f(a) = x$ and $f(b) = \{x\}$. (Such sets are used e. g. used to construct the natural numbers from some axiom systems.) Then $f^{-1}(x) = a$, $f^{-1}(\{x\}) = b$, $f^{-1}[\{x\}] = \{a\}$, $f^{-1}[\{\{x\}\}] = \{b\}$. – Jaan Jun 3 '14 at 9:32
• Or if $f$ is a function from $\{a,b\}$ to a three-element set $\{y, \{y\}, \{\{y\}\}\}$ such that $f(a) = \{y\}$ and $f(b) = \{\{y\}\}$, and we denote $x = \{y\}$, then $f^{-1}(x) = a$, $f^{-1}(\{x\}) = b$, $f^{-1}[x] = \emptyset$, $f^{-1}[\{x\}] = \{a\}$, $f^{-1}[\{\{x\}\}] = \{b\}$. (So $f^{-1}(\{x\}) \neq f^{-1}[x]$.) – Jaan Jun 3 '14 at 9:39
• There are some authors that write $f^{-1}(\{x\})$ as $f^{-1}(x)$. – Paulo Henrique Jun 5 '14 at 18:14
• I guess you meant $f^{-1}[\{x\}]$ by the notation in my comments when you wrote $f^{-1}(\{x\})$. The only consistent notation for preimages that I know of is that in my comments, and Jech & Hrbacek seem to use the same (I looked into the 1999 ed). But outside hard-core set theory one usually doesn't encounter such complex situations and so many authors are indeed a little sloppy and write ordinary brackets instead of square ones and identify one-element sets with their element. Maybe that's even not so bad. But I think that in your answer, $f^{-1}[x]$ should be changed to $f^{-1}[\{x\}]$. – Jaan Jun 5 '14 at 22:53

I can't answer in general, but in probability, square brackets are used to signal notation hacks. It basically means "please don't parse this like usual".

When someone writes $f(something)$, generally what they mean is the following:

• $f$ is a function, with well defined domain and codomain (even if these haven't been mentioned explicitly);
• Whatever is written in place of $something$ is a element from the domain, or a variable expression that can only take on values of the domain (that is, it only "evaluates" to values in the domain after the variables are bound)

A probability measure is a function that takes something from the event space (its domain) and output something in $[0,1]$. At first, it seems you could just stick to regular function notation to talk about it. Let's consider a simple bernoulli trial. Its probability measure is:

$P(a) = p\quad$ when $a=1$

$P(a) = (1-p)\quad$ when $a=0$

However, this gets stale fast. Say you have two random variables, $X$ and $Y$. The values $X$ can take are $S_X$, and the values $Y$ can take are $S_Y$. Your sample space is $S= S_X \times X_Y$. How do you write down the probability of $X$ being, say, between 1 and 2, using only regular set theory notation?

Like this:

$P([1,2] \times S_Y)$

or perhaps:

$P( {\langle x,y \rangle : x \in [1,2]})$

The problem here is that we don't care about the values $Y$ takes, but we have to explicitly mention $S_Y$ it in order to construct valid samples. It would be ideal if our notation didn't make us do it (what if we had 10 variables?). So we just use square brackets and write it like this:

$P[ X \in [1,2]]$

$X\in [1,2]$ looks like a boolean value. It, however, should be understood as a subset from the sample space ("all samples in which $X\in [1,2]$"). If you just wrote $P( X \in [1,2])$, with regular parenthesis, this could be parsed as inputting $true$ or $false$ to $P$, which makes no sense.

$P(X∈[1,2]) = P(true) = \ ? \$ ($true$ is not in the domain of $P$)

You could argue that I'm being too strict: $P( X \in [1,2])$ is perfectly reasonable if you consider $X\in [1,2]$ as a shorthand for an event. But still, you'd have to break the rules to talk about conditional probability:

$P[A|B] \triangleq \dfrac{P[A\cap B]}{P[B]}$

There's just no way to interpret $A|B$ as an event in this case. $P[A|B]$ could be different from every possible value of the original probability measure function! Consider the following probability measure:

$P(\{1\}) = 1/3$

$P(\{2\}) = 1/3$

$P(\{3\}) = 1/3$

In this case, $P(x)$ can evaluate to 0, 1/3, 2/3 or 1. Now let's consider the events $A = \{1\}$ and $B=\{1,2\}$:

$P[A|B] = 1/2$

Since we generally want this more convenient notation, we just use square brackets all the time for probability measures, as a warning that things might get weird.