# Square brackets instead of parens for functions?

Why are sometimes square brackets used to apply parameters to functions instead of the usual round parentheses?

For instance, in my probability course, they use $\text{P}[X]$ to denote the probability that some event in the set $X$ comes to pass.

$$\text{P}[X] = \sum_{x \in X} p(x)$$

Is there any rule as to when to use square brackets instead of parens or is this arbitrary?

-
Who knows? As far as I can tell, it's arbitrary. I would avoid square brackets in general. – Qiaochu Yuan Feb 26 '13 at 19:14
I use brackets for stylistic reasons, with no different meaning from parentheses, whenever I judge that too many nested parentheses would look confusing/ugly; for instance I might write $f[g(x)]$ instead of $f(g(x))$. – user7530 Feb 26 '13 at 19:14
I used to distinguish between $\mathbb{P}[\cdots]$ and $\mathbb{P}(\cdots)$: in the former, the description of the set to be measured goes between the brackets, and in the latter, the set to be measured goes between the parentheses. So, for example, $\mathbb{P}[X = x]$ but $\mathbb{P}(\{ x \})$. – Zhen Lin Feb 26 '13 at 19:16

In Jech & Hrbacek's Introduction to Set Theory, the author adopt this notation to avoid confusion about images of sets and images of elements contained in such sets. For instance, is quite common denote $f^{-1}(\{x\})$ by $f^{-1}(x)$; in the square brackets notation we'd write $f^{-1}[x]$, which is more clean than $f^{-1}(\{x\})$ and not so abusive as $f^{-1}(x)$. Other reason is sets of sets: if we consider a set $A = \{A_1,\dots, A_n\}$ and a function $f:A\to B$ it would not be didactic to write $f(A')$ for some $A'\subseteq A$, for the elements of $A$ is also denoted by capital letters.
$f^{-1}(\{X\})$ and $f^{-1}[X]$ are different things. $f^{-1}[X]$ is the preimage of a subset $X$ of the codomain and $f^{-1}(x)$ is the preimage of an element of the codomain (only if f is injective). For example, consider a function $f$ from a two-element set $\{a,b\}$ to a two-element set $\{x, \{x\}\}$ such that $f(a) = x$ and $f(b) = \{x\}$. (Such sets are used e. g. used to construct the natural numbers from some axiom systems.) Then $f^{-1}(x) = a$, $f^{-1}(\{x\}) = b$, $f^{-1}[\{x\}] = \{a\}$, $f^{-1}[\{\{x\}\}] = \{b\}$. – Jaan Jun 3 '14 at 9:32
Or if $f$ is a function from $\{a,b\}$ to a three-element set $\{y, \{y\}, \{\{y\}\}\}$ such that $f(a) = \{y\}$ and $f(b) = \{\{y\}\}$, and we denote $x = \{y\}$, then $f^{-1}(x) = a$, $f^{-1}(\{x\}) = b$, $f^{-1}[x] = \emptyset$, $f^{-1}[\{x\}] = \{a\}$, $f^{-1}[\{\{x\}\}] = \{b\}$. (So $f^{-1}(\{x\}) \neq f^{-1}[x]$.) – Jaan Jun 3 '14 at 9:39
There are some authors that write $f^{-1}(\{x\})$ as $f^{-1}(x)$. – Paulo Henrique Jun 5 '14 at 18:14
I guess you meant $f^{-1}[\{x\}]$ by the notation in my comments when you wrote $f^{-1}(\{x\})$. The only consistent notation for preimages that I know of is that in my comments, and Jech & Hrbacek seem to use the same (I looked into the 1999 ed). But outside hard-core set theory one usually doesn't encounter such complex situations and so many authors are indeed a little sloppy and write ordinary brackets instead of square ones and identify one-element sets with their element. Maybe that's even not so bad. But I think that in your answer, $f^{-1}[x]$ should be changed to $f^{-1}[\{x\}]$. – Jaan Jun 5 '14 at 22:53