4
$\begingroup$

I am bad at math and am having an extremely hard time trying to understand the conjugate of a function. I am not sure why the dotted line is the conjugate function. How do I find that? This is my limited understanding of the steps.

1) draw $y^Tx$ which is equals to $x y$ in 2 dimensions. Question: is $x$ my gradient or is $y$ my gradient? Because the original equation is $f(x)$ but now is $f^* (y)$?

2) Maximize the function $x y - f(x)$ with respect to $f$ because I am to find the supremum of the expression. How do I draw this?

So I only know how to draw the line $x y$. Then I am stuck. It would be nice if someone could give a numerical example by stepping through some x or y values ( i am not sure if i am supposed to step through the x or y values). I can only understand after seeing an example with numbers.

enter image description here

Question: Why do I shift the line downwards (blue gap) ? The red gap looks bigger to me and we are finding the maximum.

enter image description here

$\endgroup$
2
  • $\begingroup$ It may help to know that the more standard term for "conjugate" is "Legendre transform". Unfortunately, there are two conflicting conventions for the Legendre transform: one turns convex functions into concave functions, the other one turns convex functions into convex functions. So you have to be careful to pay attention to which convention you are using. $\endgroup$
    – Ian
    Mar 20, 2018 at 21:40
  • $\begingroup$ The magnitude of the red gap may be larger than the blue gap, but the conjugate function cares about the signed gap. Perhaps it helps to draw out the conjugate function for two fixed values of $y$, say $y_1$ and $y_2$. $\endgroup$
    – jjjjjj
    Oct 21, 2021 at 15:13

2 Answers 2

Reset to default
3
$\begingroup$

The dotted line is not the conjugate itself. It is a method to compute $f^*(y)$ for a given $y$. For that given value of $y$ (say $y=3$) you want to solve $\sup_x \{ 3x - f(x) \}$. The supremum is attained where the derivative of $3x - f(x)$ is $0$, i.e., $f'(x) = 3$. By moving a dotted line with slope $3$ up or down until it touches the function, you find such $x$ (a horizontal line would find $x$ such that $f'(x)=0$).

$\endgroup$
4
  • $\begingroup$ Thank you, the first 3 lines were extremely helpful. Stupid question: Why do I find the maximum by moving the line 3x up or down until it is tangent to f(x) ? I know its because you showed that the solution can be found at f'(x) = 3 but in the new picture I attached, since f*(y) is the gap it looks like I can get a bigger value by shifting it up. I hope you will answer it because I think I am very close to understanding the conjugate $\endgroup$
    – Kong
    Mar 20, 2018 at 21:38
  • 1
    $\begingroup$ The intersection with the y-axis is $-f^*(y)$, so moving it up would result in a larger value of $-f^*(y)$ / smaller value of $f^*(y)$, which is the wrong value because you are taking the supremum. When $f$ is convex, there is only way to make the lines tangent. $\endgroup$
    – LinAlg
    Mar 20, 2018 at 22:33
  • $\begingroup$ thank you very much !!! i shall go on to analyze the examples listed in p92 ocw.mit.edu/courses/electrical-engineering-and-computer-science/… $\endgroup$
    – Kong
    Mar 20, 2018 at 23:30
  • $\begingroup$ i see it now that the blue gap corresponds to the "distance" f(y). f(x) is greater then 3x at the red gap so it results in a "negative distance" $\endgroup$
    – Kong
    Mar 20, 2018 at 23:55
2
$\begingroup$

If $f(x) = x^2$, then $f^*(y) = \sup_x xy-x^2$. We see that the $\sup$ over $x$ is attained when $y-2x = 0$, so we have $f^*(y) = {y^2 \over 4}$.

If $f(x) = c$, a constant, then $f^*(y) = \sup_x xy-c$. We see that this is $+\infty$ whenever $y \neq 0$, and is $-c$ for $y = 0$. Hence $f^*(y) = I_{\{0\}}(y)- c$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .