I think that the reason you have this particular interests in partial function or even heard of the term is mostly because you have read recursion theory.
To elaborate, in most other areas of mathematics, function are by definition totally defined on their domain. To say that $f : X \rightarrow Y$ but $f$ is not define on all of $X$ is not a usual mathematical practice. Most people would restrict the domain so that $f$ is totally defined. For example, the function $f(x) = \frac{1}{x}$ where $x$ comes from some field. $f : \mathbb{R} \rightarrow \mathbb{R}$ then $f$ would be partial. Most people would likely modify this to say $f : \mathbb{R} - \{0\} \rightarrow \mathbb{R}$. In other areas, people would likely be careful to specify the domain and range.
Partial functions do appear in other areas of mathematics. Qiaochu Yuan seem to mention the study of the poles of complex value functions. For example, the residue formula gives some useful information about complex value functions using the poles.
However, in computability theory or recursion theory, there is a useful significance to introduce total and partial functions. In computability theory, you study subsets of the natural numbers and function defined on these subset. Because these function and sets correspond to Turing Machine and algorithms, their inputs (for simplicity) can be coded as natural numbers. By specifying the domain of function, in the context of recursion theory, to be $\omega$, represents the intuitive idea of an algorithm. Moreover, many computer programs, Turing machines, or algorithm used in real practice don't terminate on all inputs. Because recursion theory is the study of the computational aspect of sets, it makes sense to include these sort of partial functions.
The above is sort of an intuitive idea of why recursion theory (the study of computation) should naturally consider partial functions because algorithms naturally don't halt on all input.
Moreover, partial functions play a major role in recursion theory that is not seen in many other areas of mathematics. The partial functions are essential to computability theory. First, you are aware of the enumeration theorem for partial computable functions (i.e. the universal Turing Machine exists). Almost every theorem in computability theory uses this fact. Moreover, it can be prove that there there is no effective enumeration of the computable functions. This enumeration is absolutely necessary in many of the construction such as finite injury arguments. Moreover, the c.e. sets play a very important role in computability theory. They are defined to be the domain of the partial computable functions. Many natural problems in mathematics such that Halting Problem, diophantine equations, etc have a correspond sets that is c.e.
The term partial function is more ubiquitous in computability theory because the most fundamental objects of the theory the enumeration theorem and the c.e. sets are naturally expressed using them.