# Composition of convex function and affine function

Let $g: E^{m} \rightarrow E^{1}$ be a convex function, and let $h: E^{n} \rightarrow E^{m}$ be an affine function of the form $h(x)=Ax+b$, where $A$ is an $m \times n$ matrix and $b$ is an $m \times 1$ vector. Then, show that the composite function $f : E^n \rightarrow E^{1}$ defined as $f(x)=g(h(x))$ is a convex function.

Also, assuming twice differentiability of $g$, derive the expression for the hessian of $f$

Let $0 < \theta < 1$ and $x_1, x_2 \in E^m$. Note that $h(\theta x_1 + (1-\theta)x_2) = \theta h(x_1) + (1-\theta)h(x_2)$. It follows that \begin{align} f(\theta x_1 + (1-\theta) x_2) &= g(\theta h(x_1) + (1-\theta)h(x_2)) \\ &\leq \theta g(h(x_1)) + (1-\theta) g(h(x_2)) \\ &= \theta f(x_1) + (1-\theta) f(x_2) \end{align} so $f$ is convex.
From the chain rule, $f'(x) = g'(h(x)) h'(x) = g'(h(x))A$ so \begin{align} \nabla f(x) &= f'(x)^T \\ &= A^T g'(h(x))^T \\ &= A^T \nabla g(h(x)). \end{align} The chain rule again now tells us that $\nabla^2 f(x) = A^T \nabla^2 g(h(x)) h'(x) = A^T \nabla^2 g(h(x)) A$.
• Thank you very much. Can you explain why $$\nabla f(x) &= f'(x)^T$$ Jan 28 '14 at 11:35
• If $f:\mathbb R^n \to \mathbb R^m$ is differentiable at $x$, then $f'(x)$ is an $m \times n$ matrix such that $f(x + \Delta x) \approx f(x) + f'(x) \Delta x$ when $\Delta x$ is small. So if $f:\mathbb R^n \to \mathbb R$, then $f'(x)$ is a $1 \times n$ matrix (row vector). In optimization, the convention is that $\nabla f(x)$ is a column vector, so $\nabla f(x) = f'(x)^T$. Also, $\nabla^2 f(x) = q'(x)$, where $q(x) = \nabla f(x)$. (So $q:\mathbb R^n \to \mathbb R^n$.) Jan 28 '14 at 18:04