# Notation for image and preimage

Let $X$, $Y$ be sets and $f:X\rightarrow Y$ be a map. Denoting the image of $D\subset X$ under $f$ by $f(D)$ can sometimes be confusing. As for preimages, I've seen unambiguous notation like $f^*\mathcal{O}$, where $\mathcal{O} \subset \mathcal{P}(Y)$. (This is also an example of the "confusing" notation of an image, though). For images, is analogous notation $f_*D$ for denoting the image of $D\subset X$ under $f$ used in the literature?

-
I have seen the super- and subscripted asterisk used for images and preimages. The problem, at least for me, is that I never could remember which one is which. Because of this, I think the bracket notation, as mentioned by Asaf, is better. – Miha Habič Feb 16 '12 at 9:47
@Miha: The principle is quite easy: $f^{-1}$ is written with a superscript, and so is $f^*$. More generally, in category theory, superscripts indicate contravariance, and the inverse image map is the result of applying the contravariant powerset functor. I have seen $f_*$ once, but the author was a category theorist... (I would write $f_!$ myself, since it's left adjoint to $f^*$, not right adjoint!) – Zhen Lin Feb 16 '12 at 10:05
@Miha: The asterisks appeared in the introductory courses on linear algebra and general topology so frequently that I am no longer unsure which one is which. – Pteromys Feb 16 '12 at 10:38

Mac Lane and Birkoff in their Algebra use $f^*$ and $f_*$. Categorical notions permeate this book and the notation is consistent with the two functors image and inverse image on the category of sets.

-

In my neck of the woods a common notation for $\{f(x)\mid x\in D\}$ is $f[D]$, and in some places you can also find people using the analysts-confusing double-prime, that is $f''D$.

For the preimage, the principle is the same: $\{x\mid f(x)\in O\}=f^{-1}[O]$.

When we teach this in the introductory course we say that if $D=\{x\}$ then we write $f[x]$ instead of $f[\{x\}]$ and similarly $f^{-1}[y]$ instead of $f^{-1}[\{y\}]$. The brackets remain to distinguish sets from elements.

-
There are also set-theoretic contexts in which the double prime is used without the square brackets, e.g., $f''x$ for $\{f(y):y\in x\}$. – Brian M. Scott Feb 16 '12 at 9:46
@Brian: Indeed I meant for that. – Asaf Karagila Feb 16 '12 at 9:51
@Asaf: The proper notation is to use $f “ x$ (so as to avoid confusing analysts). According to Thomas Forster this goes back to Russell and Whitehead. – Zhen Lin Feb 16 '12 at 10:03
@Zhen Lin: Oops, you’re right; I was careless when I wrote $f''x$. – Brian M. Scott Feb 16 '12 at 10:06
@Zhen: I have seen both in use, and the one appearing in my answer I recall seeing more often. – Asaf Karagila Feb 16 '12 at 13:41

There are a number of notations for the image, Lawvere and Rosebrugh list five different notations in their book "Sets for Mathematics", pg 137:

$$\mathcal{P}f,\ f_!,\ f[\ ],\ \exists_f,\ \textrm{im}_f$$

The asterisk notation can be explained through topos theory. The function $f:X \to Y$ gives rise to a geometric morphism $f: \mathbf{Set}/X \to \mathbf{Set}/Y$. Every geometric morphism $f:\mathcal{E} \to \mathcal{F}$ contains an adjunction $(f^* \dashv f_*)$, called the inverse image and direct image respectively. Some geometric morphisms have extra adjoints, the notation goes:

$$f_! \dashv f^* \dashv f_* \dashv f^!$$

Functors with an exclamation mark (!), usually called shriek functors, are not present in every geometric morphism. Subscript functors follow the geometric morphism, so $f_!, f_*: \mathcal{E} \to \mathcal{F}$, while superscript functors go in the opposite direction, $f^*, f^!: \mathcal{F} \to \mathcal{E}$.

If we return to a set function $f:X \to Y$, the corresponding geometric morphism $f$ has an inverse image $f^*$, which takes the pullback along $f$, and direct image $f_* = \Pi_f$, which is a bit harder to describe. But there is an extra adjoint, $f_! = \Sigma_f$, which composes with $f$. This adjoint, when restricted to subobjects of the terminal object, corresponds to the image, not $\Pi_f$.

Following this convention the notation $f_*$ should not be used for the image, as the image is left-adjoint to the inverse image, as Zhen Lin says above. However there is a functor $f_*$, which on subsets has the definition

$$f_*(U) = \{ y \in Y\ |\ \forall x \in X, f(x) = y \to x \in U \}$$

This does not get as much use as the image. I would still use $f_!$ for the image. Whatever notation you use make it explicit what you mean, as not everyone will recognise this notation.

-
+1, excellent first answer. But you should be aware that the functors $f_!$ and $f^!$ are traditionally called ‘shriek’, not ‘exclamation’. – Zhen Lin Feb 16 '12 at 22:31
Thanks for that, I have added a comment about that terminology. – Boris Y Feb 17 '12 at 0:47