How to interpret physically conjugate functions? If given a convex function $f: \mathbb{R} \to \mathbb{R}$, then the conjugate function $f^*$ is defined as $$f^*(s) = \sup_{t \in \mathbb{R}} (st-f(t))$$
Now i want to understand what is the physical interpretation of this conjugate function? What is its exposition? Please help me. 
 A: The basic idea behind duality in convex analysis is to view a (closed) convex set $C$ as an intersection of half spaces.  Applying this idea to the epigraph of a convex function $f$ suggests that we should view $f$ as a supremum of affine functions.
An affine minorant of $f$ is a function $x \mapsto \langle m, x \rangle - b$ such that
$$
\tag{$\spadesuit$} f(x) \geq \langle m, x \rangle - b \quad \text{for all } x.
$$
The vector $m$ is called the "slope" of the affine minorant.
Typically $f$ has many affine minorants with a given slope $m$, corresponding to different values of the scalar $b$.  We only care about the best affine minorant with slope $m$ --- in other words, we only care about the best scalar $b$.  So: For a given $m$, which value of $b$ is the "best"?  Which value of $b$ makes the inequality in $(\spadesuit)$ as tight as possible?
Notice that
\begin{align}
& f(x) \geq \langle m, x \rangle - b \quad \text{for all } x \\
\iff & b \geq \langle m, x \rangle - f(x) \quad \text{for all } x\\
\iff & b \geq \sup_x \, \langle m, x \rangle - f(x) = f^*(m).
\end{align}
This shows that the best choice of $b$ is $f^*(m)$.  We have just discovered the convex conjugate $f^*$.  The whole point of $f^*$ is that it tells us how to view $f$ as a supremum of affine functions.  You give $f^*$ a slope $m$, and it gives you the best choice of $b$.
It now becomes very intuitive that $f$ can be recovered from $f^*$, because:
\begin{align}
f(x) &= \sup_m \, \langle m, x \rangle - f^*(m) \quad \text{(because $f$ is a supremum of affine functions)} \\
&= f^{**}(x).
\end{align}
While it is obvious that $f$ can be recovered from $f^*$, the fact that the "inversion formula" $f = f^{**}$ is so simple is a surprising and beautiful fact.
I wrote a similar explanation with some more details here:
Please explain the intuition behind the dual problem in optimization.
A: Suppose $f: \mathbb{R}^n \to \mathbb{R}$, then the conjugate function is:
$$f^*(\mathbf{y}) \triangleq \sup_{x\in \text{dom} f} [\mathbf{y}^ \mathsf{T}\mathbf{x}-f(\mathbf{x})],$$
where: $\text{dom}f^* = \{\textbf{y}|f^*(\mathbf{y})<+\infty\}$. Then, it is clear from the definition that it is a function bounded above.
Some notions and descriptions about the conjugate function:

*

*Even if $f(x)$ is a non-convex function, then the conjugate function $f^*(\mathbf{y})$ is guaranteed to be a convex function in $\mathbf{y}$.
[Notice: The conjugate function is the pointwise supremum of a set of affine functions in $\mathbf{y}$ (here, convexity aspect of affine function matters) , so, it is convex.]


*As an interpretation, it somehow represents (encodes) the convex hull of the epigraph of $f$ in terms of its supporting hyperplanes (The concept behind this is depicted in figure below.).


*There is an alternative way to obtain the (Lagrange) dual function of the primal problem by using conjugate function.


*If the function $f$ is closed and convex, then $f^{**} = f$ (conjugate of conjugate is function itself).


*Let $g_{f}$ be the convex envelope of $f$. If $f$ is a lower semi-continuous function and its domain is compact (i.e., closed and bounded), then $f^{**} = g_{f}$ [4].

References for more info.:

*

*Convex Optimization for Signal Processing and Communications, by Chong-Yung Chi.

*Convex Optimization by Stephen Boyd.

*Convex analysis and optimization, by D. P. Bertsekas.

*Lagrange multipliers and nonconvex programs, by J. Falk.

