9
$\begingroup$

In the equation:

$f_\theta(x)=\theta_1x$

Is there a reason that $\theta$ might be a subscript of $f$ and not either a second parameter or left out of the left side of the equation altogether? Does it differ from the following?

$f(x,\theta)=\theta_1x$

(I've been following the Machine Learning class and the instructor uses this notation that I've not seen before)

$\endgroup$
1
  • 2
    $\begingroup$ The notations are equivalent, but using a subscript sort of suggests that it's fixed for most of the discussion, and $x$ is the one that's changing. Leaving it out means for sure that it's fixed for the discussion (i.e., all the $f$'s you see should be taken with the same $\theta$.) $\endgroup$
    – Ted
    Oct 5, 2011 at 5:38

2 Answers 2

12
$\begingroup$

As you note, this is mostly notational choice. I might call the $\theta$ a parameter, rather than an independent variable. That is to say, you are meant to think of $\theta$ as being fixed, and $x$ as varying.

As an example (though I am not sure of the context you saw this notation), maybe you are interested in describing the collection of functions $$f(x) = x^2+c$$, where $c$ is a real number. I might call this function $f_c(x)$, so that I can later say that for $c\leq 0$, the function $f_c$ has two real roots, while for $c>0$ the roots are uniformly convex. I think these statements would be much more opaque if I made them about the function $f(x,c)$.

$\endgroup$
1
$\begingroup$

I personally dislike the notation. And I believe it means different things in different contexts. For example, it means the opposite of what the other answers say in probability, see:

https://en.wikipedia.org/wiki/Likelihood_function#Discrete_probability_distribution

It is shown that $P_{\theta}(X = x)$ and $p_{\theta}(x)$ both mean that $X/x$ is the constant variable in this context and we are plotting with respect to $\theta$.

$\endgroup$
2
  • $\begingroup$ I think it makes sense in probability. $p_{\theta}(x)$ denotes a probability density function on $x$, which is parametrised by some "model parameters" $\theta$. It's a probability distribution on $x$, not on $\theta$ ($\int_{-\inf}^{\inf} p_{\theta}(x d\theta \neq 1$). Treated as a probability distribution it makes sense to think of $\theta$ as parameters and $x$ as the argument on which the distribution is defined. But in inference, you are interested in finding the value of $\theta$ that maximises the distrbution for some $x$, which is why you often vary and plot $\theta$ instead. $\endgroup$
    – Marses
    Aug 25, 2020 at 17:12
  • $\begingroup$ That's why in inference you often write it as a likelihood, $L(\theta) = p_{\theta}(x)$, because in inference the result of your random variable ($x$) is often "measured" and fixed, and you want to find $p_{\theta}(x)$ at different values of $\theta$. If you write $p(x, \theta)$, unless you normalise it again, you won't have a probability distribution anymore (if you can even normalise it). Take the standard normal distribution: if you treat the mean as an argument, you can't say $\int\int\frac{1}{\sqrt{2\pi}}e^\frac{(x-\mu)^2}{2} dxd\mu = 1$. In fact you can't even normalise that. $\endgroup$
    – Marses
    Aug 25, 2020 at 17:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .