From this article:

$\ldots$a maximum-a-postiori "a maximum-a-posteriori $(MAP_{x,k}^{\,\,\,\,\,1})$ estimation, seeking a pair $(\hat{x}, \hat{k})$ maximizing: $$p(x, k\mid y) \propto p(y\mid x, k)p(x)p(k).$$

What does the symbol '∝' mean in this context?

  • 1
    $\begingroup$ I think Seyhmus Güngören's answer is vague. It's just how this question is usually answered, and I think it bears elaboration. $\endgroup$ – Michael Hardy Mar 13 '13 at 2:26
  • $\begingroup$ Bernoulli used it to mean "$=$". $\endgroup$ – Pedro Tamaroff Mar 13 '13 at 3:05
  • $\begingroup$ Another question, are "x hat" and "k hat" the normalized version of vector x and k, having length 1? And why does this make sense? $\endgroup$ – user1095340 Mar 13 '13 at 10:26
  • 1
    $\begingroup$ $\hat x$ and $\hat k$ are the values of $x$ and $k$ that make $p(x,k\mid y)$ as large as possible. $\endgroup$ – Michael Hardy Mar 13 '13 at 17:07

It means proportional as a function of two variables $x$ and $k$ with $y$ fixed.

You have a prior probability density as a function of $x$ and $k$, which is just a product of a function of $x$ and a function of $k$, so that the random variables involved are independent in the prior distribution. Then new data arrives: a random variable is observed to be equal to a number $y$. The conditional density of that random variable given that the first two were equal to $x$ and $k$, is the first factor in the expression on the right.

The expression on the left is the conditional density of the random variables corresponding to $x$ and $k$ given that observed value equal to $y$.

Since it means proportional as a function of $x$ and $k$ with $y$ fixed, you need to multiply it by a normalizing constant that may depend on $y$ but does not depend on $x$ and $k$, in order to make it a probability density function, as a function of $x$ and $k$. "Constant" in this case would mean not depending on $x$ and $k$.

"Constant" always means not depending on something. Usually it's clear from the context what the "something" is, but I think it should be stated explicitly more often than it is in present-day conventional practice. Here's a favorite example of mine, involving differentiation of exponential functions: \begin{align} \frac{d}{dx} 2^x & = \lim_{h\to0}\frac{2^{x+h}-2^x}{h} \\[10pt] & = 2^x\lim_{h\to0}\frac{2^h-1}{h}\tag{1} \\[10pt] & = (2^x\cdot\text{constant})\tag{2}. \end{align}

In $(1)$, the factor $2^x$ can be taken out of the limit because it's "constant" but "constant" means not depending on $h$.

In $(2)$, "constant" means not depending on $x$.

Some instructors in calculus classes actually present this proof without mentioning the contextual change in the meaning of "constant".

Later note in response to comments below: The linked paper uses a rather obnoxious notation, $p(x)$ and $p(k)$ for the probability density functions of two different random variables. One should distinguish between capital $X$ and lower-case $x$ in expressions like $\Pr(X=x)$, where capital $X$ is a random variable and lower-case $x$ is a particular value that $X$ might be equal to. Then, if one writes $p_X(x)$ for the value of the probability density function of a random variable (capital) $X$ at the point (lower-case) $x$, then one knows that $p_X(3)$ means something different from $p_Y(3)$.

But at any rate, $p(x)p(k)$ is the notation used in the linked paper for the joint density function of a pair of independent random variables, and $p(y\mid x,k)$ is the conditional density of another random variable given the values of those two. The idea is that if you multiply those, what you get is proportional, as a function of $x$ and $k$ with $y$ fixed, to the conditional density function of the random variables corresponding to $x$ and $k$, given an observed value of the random variable corresponding to $y$.

  • $\begingroup$ Thanks. Oh, and btw. Could you answer my comments on your answer? math.stackexchange.com/questions/323000/… $\endgroup$ – user1095340 Mar 13 '13 at 10:30
  • $\begingroup$ Since, I don't know much about statistics, I would like to ask you: Could you go through it, step-by-step, referring to which part of the formula you are talking about. $\endgroup$ – user1095340 Mar 13 '13 at 16:23

The symbol you've written means propotional. Check:


  • 1
    $\begingroup$ But does it mean proportional as a function of $x$ with $k$ and $y$ fixed, or proportional as a function of $y$ with $k$ and $x$ fixed, or proportional as a function of $k$ with $x$ and $y$ fixed? My guess is none of the above: it means proportional as a function of two variables $x$ and $k$ with $y$ fixed. $\endgroup$ – Michael Hardy Mar 13 '13 at 2:15
  • $\begingroup$ Thanks, now, how to interpret it, if you want to write it as one sentence? So, lets say if you want to explain it to a non-mathperson. $\endgroup$ – user1095340 Mar 13 '13 at 9:51
  • 1
    $\begingroup$ It means right and left hand side of the equation behaves similarly. Can happen that there is a simple constant factor. For each equality one can use this term. Additionally whenever this term is used, left hand side can be obtained from the right hand side by just a simple scaling. Examples: "If an object travels at a constant speed, then the distance traveled is directly proportional to the time spent traveling, with the speed being the constant of proportionality." "The circumference of a circle is directly proportional to its diameter, with the constant of proportionality equal to $\pi$. $\endgroup$ – Seyhmus Güngören Mar 13 '13 at 10:02
  • 1
    $\begingroup$ It is MAP. So the difference between ML is the a priori probabilities, which are taken As some scaling factors in MAP. Therefore $p(x)$ and $p(k)$ in this context are jointly the scaling factor. $\endgroup$ – Seyhmus Güngören Mar 13 '13 at 18:09
  • 1
    $\begingroup$ It might also happen that he is indicating the independecy between $T$ and $X$. One should see more about the content $\endgroup$ – Seyhmus Güngören Mar 13 '13 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.