# Legendre transform of a scalar function - getting stuck at inverting vectors

I'm interested in finding the Legendre transform of the following function $$f(x) = - \log(\langle a,x\rangle)$$ where $$a$$ and $$x$$ are both $$n$$-dimensional (column) vectors. The values of $$x$$ and $$a$$ are such that the inner product is always positive

I wish to find the Legendre transform of $$f(x)$$. My attempt is below $$\langle p,x\rangle - f(x) = \langle p,x\rangle + \log(\langle a,x\rangle)$$ The gradient is given as $$\frac{d}{dx} = p + \frac{a}{\langle a,x\rangle}$$ And the hessian is $$\frac{d^2}{dxdx^{T}} = -\frac{aa^{T}}{\langle a,x\rangle^2}$$ (I think). This hessian is negative, which means the fixed point we find is a maximum - all good so far.

From here I would be like to be able to write something like the following $$f^{\ast}(p) = -\log(\langle a,x^{\ast}\rangle)$$ where $$x^{\ast} = g(p)$$, which I find by inverting that first derivative (when it is at a minimum). However, I don't know how to do this. $$\frac{a}{p} = a^{T}x$$ The thinking here is to somehow invert $$a$$, but in general one can't invert vectors in matrix multiplication, since inverses only exist for square matrices. What, then, should I do to attack this problem?

The other classic way to solve this problems is to invert the gradient of the function, i.e., to find the following function $$(\frac{d}{dx}f)^{-1}(p)$$ but since this gradient ($$\nabla f$$) is again a vector (of partial derivatives), I am again stuck on how to proceed

Assume the $$a \neq 0$$. The domain of $$f$$ is $$\mathcal D = \{x : \langle a, x\rangle > 0\}$$ and the Legendre transform is $$f^*(p) = \sup_{x \in \mathcal D} \langle p, x \rangle + \log \langle a, x\rangle$$.

Suppose it is not true that $$p = - \alpha a$$ for some $$\alpha \geq 0$$, then there exists an $$x$$ with $$\langle a, x \rangle > 0$$ and $$\langle p, x \rangle > 0$$. By increasing the magnitude of this $$x$$ arbitrarily you see that $$f^*(p) = \infty$$.

On the other hand, if $$p = -\alpha a$$ for $$\alpha \geq 0$$, then $$f^*(-\alpha a) = \sup_{x \in \mathcal D} -\alpha \langle a, x \rangle + \log \langle a, x\rangle = \log(1/\alpha) - 1$$ The reason you encountered trouble is because $$\nabla f(\mathbb R^n) \neq \mathbb R^n$$. In fact, it lives in a subspace. Thinking about the geometric definition of the Legendre transform may be helpful here.

• Sorry for the delay in replying, its taken me a while to really understand and internalise your argument and solution, but now it makes perfect sense to me, many thanks! Commented Apr 20, 2021 at 16:23

We have $$a \ne 0$$ since we need $$\langle a, x \rangle \ne 0$$.

$$f^*(p)=\sup_{x: (\langle x, a\rangle > 0) } (\langle p, x \rangle +\log \langle a, x \rangle)$$

Differentiating, we have $$\nabla_x(f)= p + \frac{a}{\langle a, x \rangle}=0$$

$$p=-\frac{a}{\langle a, x \rangle}$$

• If $$p$$ is not a negative multiple of $$a$$, let $$x = k(|p|a+|a|p)$$ where $$k>0$$ is a constant to be determined:

$$\langle p, x \rangle = k(|p|\langle p, a\rangle + |a| |p|^2)=k|p|(\langle p, a\rangle + |a||p|)$$ which is nonnegative by Cauchy-Schwarz inequaltiy.

Similarly, $$\langle a, x \rangle = k(\langle a, |p| a + |a| p\rangle=k(|p||a|^2 + |a|\langle a, p \rangle)=k|a|(|p||a| + \langle a,p \rangle )$$

which is positive by Cauchy-Schwarz inequality. We can choose $$k$$ to be arbitrarily large to make $$\langle a, x \rangle$$ to exceed $$1$$ and hence $$\langle p, x \rangle, + \log \langle a, x \rangle$$ can be chosen to go to $$\infty$$.

• If $$p$$ is a negative multiple of $$a$$, $$p =-ka$$, $$k>0$$, $$k = \frac1{\langle a, x \rangle}$$, $$\langle a, x \rangle =\frac1k$$ and $$\langle p, x\rangle=\langle -ka, x\rangle =-1.$$ That is $$f^*(-ka) = -1 - \log k$$.