Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm studying statistics, and in a lot of textbooks, the regression formulas always refers to the functions themselves as f(x) parameterized by a,b,c or something. And they are often written $f(x; a,b,c)$, why can't we just write $f(x, a, b, c)$?

I would like to know perhaps the history of how this came about, and why there is this seemingly dual method of expressing the same idea. Is there some esoteric mathematical notation that a noob like me hasn't encountered yet, or did mathematicians of old just like doing things in different ways?

An example: A function that maps age of tree to height:

$$f(age; growth\_rate) = age * growth\_rate$$ vs $$f(age, growth\_rate) = age * growth\_rate$$

share|cite|improve this question
    
Consider also sequences and time series -- it's strange to express a sequence as $a(n)$, even though it's perfectly valid -- a sequence is a function from $\mathbb{N}$ to whatever the domain of $a$ happens to be. Similarly for time series -- common to see functions like $V_t (x)$ which are actually just $V(x, t)$, or even $V(x; t)$, to come full circle. – MichaelChirico Mar 1 at 19:19
up vote 22 down vote accepted

Strictly speaking you're right. It's sometimes difficult to explain clearly what is a parameter and what is a variable. When you write $f(x;a,b,c)$, there is no mathematical difference between $x$ and $a$ for example, they're both variables for the function $f$. I think the real difference is in how we interpret things. When we have an expression such as $f(x;\theta)$ or $f_\theta(x)$, $x$ and $\theta$ don't exactly play the same role, that's why we use different notations for them. In fact it's for the sake of clarity. When you have lots of different variables in an expression, using different notations is an easy way to know what role they play.

In the framework of statistics, the notion of parameter is even more important. As you may know, there's a whole branch of statistics called parametric statistics, where we introduce families of probability distributions, indexed by a set of parameters: $$\mathcal{P}=\{P_\theta,\theta\in\Theta\}$$ where $\Theta$ is some subset of $\mathbb{R}^d$. Note that you could actually see this as a function from $\Theta$ to the set of probability distributions on a certain space. In many cases, $P_\theta$ has a density w.r.t. the Lebesgue measure, and we denote it by $f(x;\theta)$ or $f_\theta(x)$. You see in this example that $x$ and $\theta$ have very different meanings. $\theta$ is the parameter that comes from our statistical model, whereas $x$ is a variable of integration.

In other cases, we use the "opposite" notation. For example when you consider the likelihood function $l(\theta;x_1,\dots,x_n)$, we consider it primarily as a function of $\theta$ (that we want to maximize most of the time), and $x_1,\dots,x_n$ as fixed parameters (they are the observation, we can't change them). Once again, it's a question of context and meaning of the variables.

share|cite|improve this answer
    
Exactly; TL;DR: because we have very specific interpretations for how to think about parameters as opposed to variables. We tend to think of parameters as "fixed" (even though they're not, which is why we leave them as symbols). – MichaelChirico Mar 1 at 19:16
1  
If this means anything, I'd say that parameters are somewhat "more fixed" than variables. They are not the first thing that we want to make vary. – Augustin Mar 1 at 19:26
    
If you have any background in programming, a good comparison would be template/generic arguments vs function arguments. The template arguments are still technically arguments you pass to the function, but they effectively change the template of the function you're calling in the first place. – Blindy Mar 2 at 4:58

While the answers already given are correct, I suspect that they may be bringing in terminology and notations you are unfamiliar with. If so, let me try to recast the answer more simply.

You are correct that $f(x; a, b, c)$ is the exact same thing as $f(x, a, b, c)$. But the difference is in how we are using $x$ vs how we are using $a, b, c$. With $f(x, a, b, c)$, all four values are equally important, equally of interest. With the former, we are considering $f$ as a function of $x$ only. $a, b, c$ are simply values that we use in defining that function. By making these values variables too, the mathematics we do applies to all the different functions of $x$ that are defined by the various possible values of $a, b, c$, instead of having to work each particular example we need separately.

For example, when talking of linear functions, we will write things such as $$y = mx + b$$ You could consider this as defining $y$ as a function of three variables, $x, m, b$, but what is important is the dependence of $y$ on $x$. The slope $m$ and intercept $b$ are just values expressing which line is represented by the relationship between $y$ and $x$. So if want to find the root for $y = 0$, that is $x = \frac{-b}{m}$, not $b = -mx$ or $m = \frac{-b}x$.

That is what the notation $f(x; a, b, c)$ means. It is just telling you that $x$ is the one whose variance is important to us. The others we are going to consider fixed (although not currently known) values.

share|cite|improve this answer
    
I like this better than the accepted answer. If we define $f(x;a,b,c)$, we are defining a family (parameterized by $a$, $b$, and $c$), of functions of a single variable (which we represent as $x$). $f(x;1,2,3)$ is then a specific function of one variable. While this is technically the same as saying $f(x;a,b,c)$ is a function of four variables, it is not as useful a conceptualization. Given $m$ and $b$, $f(x)=mx+b$ is a linear function of one variable. If we're studying lines, we want to say that, not that $f(x,m,b)$ as a function of three variables that describes a line if we fix $m$ and $b$. – Steve Kass Mar 2 at 17:06

In fact there is a formal distinction between parameter and argument, which captures our subjective view.

Let $P$ be the parameter space (set of growth rates), $X$ the domain (set of ages) and $Y$ the codomain (set of heights). A family of functions from $X$ to $Y$ parameterised by $P$ is formally a function $f: P \rightarrow Y^X$, where $Y^X$ is the set of functions from $X$ to $Y$.

In other words, it is a function taking growth rates and returning maps from ages to heights.

Of course, you are correct that this naturally and uniquely corresponds to a function $F: P \times X \rightarrow Y$, that is, a function taking a growth rate and an age and returning a height. This natural isomorphism is succintly denoted by

$$ \left( Y^X \right)^P \approx Y^{P \times X}$$

share|cite|improve this answer
1  
See Currying. It still leads us to ask why mathematicians don't use notation like $f(a,b,c)(x)$, which I think is best explained in section 2 of this essay. In short, functions were originally meant to be of type $\mathbb R \to \mathbb R$ or $\mathbb C \to \mathbb C$. Mathematicians were wary to generalize this because it was difficult to explain the modern notion of function in terms of existing concepts. – James Wood Mar 1 at 23:46
    
But they do, except they always prefer the clearer alternative $f_{a,b,c}(x)$. Also, thanks for the nice read! – filipos Mar 2 at 23:34

Think to something like this: $f(x;\sigma) = e^{-x^2/\sigma^2}$.

If you read for example "the derivative of $f$" probably you immediately undestand that we are speaking of $\partial f/\partial x$ because you recognize $f$ as a function of $x$, whose definition contains some $\sigma$ which is not a "variable".

Otherwise if you read: $f(x, \sigma) = e^{-x^2/\sigma^2}$ a sentence like "the derivative of $f$" would be ambiguous: it's $\partial f/\partial x$? $\partial f/\partial \sigma$? $\nabla f = (\partial f/\partial x, \partial f/\partial \sigma)$?

So without this distinction every time one has to write explicitly "the derivative of $f$ with respect to the variable $x$".

share|cite|improve this answer

The area where I am used to seeing parameterizations is when considering curves and similar geometric objects. Consider a curve drawn in $\mathbb{R}^3$, this is just a set of points and initially is not a function of any kind. But we can in fact make it into the image of a function by defining a mapping $f:[0,1] \rightarrow \mathbb{R}^3$ which 'traces' the curve as the input variable goes from 0 to 1. So in this context a parameterization means taking an object that is just a subset of a given space and defining a mapping whose image is the object we are interested in. The inportant thing here is that the choice of this mapping is arbirtrary, in that there are an infinite number of different functions we could define whose image would be our object of study. Hence calling it a parameterization really is indicating that we have chosen a suitable functional description, but indicates that it is not unique.

This seems the same as for the function example you give, as there are any number of ways of taking the heights of trees and defining functions which map to these correctly for each tree. So your semicolon notation appears to simply be saying that this function has been chosen to use growth rate along with the fixed age input in order to define height, but is indicating that there were other choices you could have made instead.

share|cite|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.