5
$\begingroup$

Let $\phi:\Omega\rightarrow\mathbb{R}$ ($\Omega \subset\mathbb{R}^n$) be a continuously-differentialble real-valued and strictly convex function defined on a closed convex set $\Omega$. For each two points the Bregman divergence is defined as:

$$D_\phi(x,y)=\phi(x)-\phi(y)-<\nabla \phi(y),x-y>$$

If we write the Bregman divergence for $y$ and $z$

$$D_\phi(y,z)=\phi(y)-\phi(z)-<\nabla \phi(z),y-z>$$

We have the following identity

$$D_\phi(x,y)+D_\phi(y,z)=D_\phi(x,z)+<\nabla \phi(z)-\nabla \phi(y),x-y>$$

Is there any geometrical interpretation to explain two sides of this equality based on the definition of the Bregman divergence and the projection of $\nabla \phi(z)-\nabla \phi(y)$ on $x-y$ or $x-y$ on $\nabla \phi(z)-\nabla \phi(y)$?

To have better understanding one can use the following picture.

enter image description here

$\endgroup$

1 Answer 1

2
$\begingroup$

It's a generalization of the identity (some call it Pythagorean identity) for Hilbertian norms,

$$\frac{1}{2}\|x-z\|^2 = \frac{1}{2}\|x-y+y-z\|^2 = \frac{1}{2}\|x-y\|^2 + \frac{1}{2}\|y-z\|^2 + \langle x-y,y-z\rangle$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.