Why do we say function "parameterized by" vs just function of (x,y,z,...)? I'm studying statistics, and in a lot of textbooks, the regression formulas always refers to the functions themselves as f(x) parameterized by a,b,c or something. And they are often written $f(x; a,b,c)$, why can't we just write $f(x, a, b, c)$?
I would like to know perhaps the history of how this came about, and why there is this seemingly dual method of expressing the same idea. Is there some esoteric mathematical notation that a noob like me hasn't encountered yet, or did mathematicians of old just like doing things in different ways?
An example: A function that maps age of tree to height:
$$f(age; growth\_rate) = age * growth\_rate$$ vs $$f(age, growth\_rate) = age * growth\_rate$$
 A: Think to something like this: $f(x;\sigma) = e^{-x^2/\sigma^2}$.
If you read for example "the derivative of $f$" probably you immediately undestand that we are speaking of $\partial f/\partial x$ because you recognize $f$ as a function of $x$, whose definition contains some $\sigma$ which is not a "variable".
Otherwise if you read:
$f(x, \sigma) = e^{-x^2/\sigma^2}$
a sentence like "the derivative of $f$" would be ambiguous: it's $\partial f/\partial x$? $\partial f/\partial \sigma$? $\nabla f = (\partial f/\partial x, \partial f/\partial \sigma)$?
So without this distinction every time one has to write explicitly "the derivative of $f$ with respect to the variable $x$".
A: In fact there is a formal distinction between parameter and argument, which captures our subjective view.
Let $P$ be the parameter space (set of growth rates), $X$ the domain (set of ages) and $Y$ the codomain (set of heights). A family of functions from $X$ to $Y$ parameterised by $P$ is formally a function $f: P \rightarrow Y^X$, where $Y^X$ is the set of functions from $X$ to $Y$.
In other words, it is a function taking growth rates and returning maps from ages to heights.
Of course, you are correct that this naturally and uniquely corresponds to a function $F: P \times X \rightarrow Y$, that is, a function taking a growth rate and an age and returning a height. This natural isomorphism is succintly denoted by
$$ \left( Y^X \right)^P \approx Y^{P \times X}$$
A: Strictly speaking you're right. It's sometimes difficult to explain clearly what is a parameter and what is a variable. When you write $f(x;a,b,c)$, there is no mathematical difference between $x$ and $a$ for example, they're both variables for the function $f$. I think the real difference is in how we interpret things. When we have an expression such as $f(x;\theta)$ or $f_\theta(x)$, $x$ and $\theta$ don't exactly play the same role, that's why we use different notations for them. In fact it's for the sake of clarity. When you have lots of different variables in an expression, using different notations is an easy way to know what role they play.
In the framework of statistics, the notion of parameter is even more important. As you may know, there's a whole branch of statistics called parametric statistics, where we introduce families of probability distributions, indexed by a set of parameters:
$$\mathcal{P}=\{P_\theta,\theta\in\Theta\}$$
where $\Theta$ is some subset of $\mathbb{R}^d$.
Note that you could actually see this as a function from $\Theta$ to the set of probability distributions on a certain space. In many cases, $P_\theta$ has a density w.r.t. the Lebesgue measure, and we denote it by $f(x;\theta)$ or $f_\theta(x)$. You see in this example that $x$ and $\theta$ have very different meanings. $\theta$ is the parameter that comes from our statistical model, whereas $x$ is a variable of integration.
In other cases, we use the "opposite" notation. For example when you consider the likelihood function $l(\theta;x_1,\dots,x_n)$, we consider it primarily as a function of $\theta$ (that we want to maximize most of the time), and $x_1,\dots,x_n$ as fixed parameters (they are the observation, we can't change them). Once again, it's a question of context and meaning of the variables.
A: The area where I am used to seeing parameterizations is when considering curves and similar geometric objects. Consider a curve drawn in $\mathbb{R}^3$, this is just a set of points and initially is not a function of any kind. But we can in fact make it into the image of a function by defining a mapping $f:[0,1] \rightarrow \mathbb{R}^3$ which 'traces' the curve as the input variable goes from 0 to 1. So in this context a parameterization means taking an object that is just a subset of a given space and defining a mapping whose image is the object we are interested in. The inportant thing here is that the choice of this mapping is arbirtrary, in that there are an infinite number of different functions we could define whose image would be our object of study. Hence calling it a parameterization really is indicating that we have chosen a suitable functional description, but indicates that it is not unique.
This seems the same as for the function example you give, as there are any number of ways of taking the heights of trees and defining functions which map to these correctly for each tree. So your semicolon notation appears to simply be saying that this function has been chosen to use growth rate along with the fixed age input in order to define height, but is indicating that there were other choices you could have made instead.
A: While the answers already given are correct, I suspect that they may be bringing in terminology and notations you are unfamiliar with. If so, let me try to recast the answer more simply.
You are correct that $f(x; a, b, c)$ is the exact same thing as $f(x, a, b, c)$. But the difference is in how we are using $x$ vs how we are using $a, b, c$. With $f(x, a, b, c)$, all four values are equally important, equally of interest. With the former, we are considering $f$ as a function of $x$ only. $a, b, c$ are simply values that we use in defining that function. By making these values variables too, the mathematics we do applies to all the different functions of $x$ that are defined by the various possible values of $a, b, c$, instead of having to work each particular example we need separately.
For example, when talking of linear functions, we will write things such as $$y = mx + b$$ You could consider this as defining $y$ as a function of three variables, $x, m, b$, but what is important is the dependence of $y$ on $x$. The slope $m$ and intercept $b$ are just values expressing which line is represented by the relationship between $y$ and $x$. So if want to find the root for $y = 0$, that is $x = \frac{-b}{m}$, not $b = -mx$ or $m = \frac{-b}x$.
That is what the notation $f(x; a, b, c)$ means. It is just telling you that $x$ is the one whose variance is important to us. The others we are going to consider fixed (although not currently known) values.
