I've proved that $f(x_i): \mathbb{R} \to \mathbb{R}$ is convex (it only depends on the the vector component $x_i$); can I then say $g(x)\equiv f(x_i): \mathbb{R}^n \to \mathbb{R}$ (I just expanded the "scope" of $f$) is also convex? More generally I want to say that a function is convex in its domain element if it's convex in some "components" of the domain element that it depends on.

The context is this:

I'm trying to show that the relative entropy $R(u,v)$ is convex in the pair $(u,v)$, $u,v \in \mathbb{R}_{++}^n$. I've shown that each $h_i(u_i,v_i) = u_i \log(u_i/v_i)$ is convex in $(u_i,v_i)$, and want to argue that $R$ is convex in the pair $(u,v)$ because it's the sum $\sum_i h_i(u_i,v_i)$, each $h_i$ convex in $(u,v)$.


Now I'm pretty sure this is OK, as I my textbook at one point argued "$g(y)$ is log-concave in $(x,y)$" (Boyd's Convex Optimization, p. 106)


Denote $w=(u,v)$, where $u,v\in\mathbb{R}_{++}^{n}$. Also, I write $h_{i}(w)$ with the understanding that $h_{i}$ depends only on $i$th entry in $w$.

With this notation you have already proven that $$h_{i}(\alpha w+(1-\alpha)w')\leq\alpha h_{i}(w)+(1-\alpha)h_{i}(w')$$ for all $i$. The result you are after is, given $R(w)=\sum_{i}h_{i}(w)$, $$R(\alpha w+(1-\alpha)w')\leq\alpha R(w)+(1-\alpha) R(w').$$ The latter inequality holds since you get it as the sum of the former inequality.

  • $\begingroup$ Thanks. What I wanted to ensure is the equivalence that $h_i$ is convex in $w$ iff $h_i$ is convex in $w_i$, but I guess this is obvious since $h_i$ only uses the $i$th component of $w$. One more thing: what's the best way to think about the domain of $R$? As $\mathbb{R}^n \times \mathbb{R}^n$? or $\mathbb{R}^{2n}$? $\endgroup$ – ladiesman Feb 2 '17 at 2:37
  • $\begingroup$ Aren't $\mathbb{R}^{n}\times\mathbb{R}^{n}$ and $\mathbb{R}^{2n}$ equivalent? $\endgroup$ – Jan Feb 2 '17 at 2:48
  • $\begingroup$ I was taught that technically these two vector spaces are isomorphic, but not equal, like one is a vector of tuples, the other a vector of numbers (see Axler's Linear Algebra Done Right p.92 example 3.74). But I suppose we can use them interchangeably $\endgroup$ – ladiesman Feb 2 '17 at 3:53

I will answer your very first question. Consider $f(x) = x^\top A \, x$ for some matrix $A \in \mathbb{R}^{n\times n}_{\textrm{sym}}$. Then,

  • $f$ is convex iff $A$ is positive semi-definite
  • $f$ is convex w.r.t. the variable $x_i$, iff $A_{ii} \ge 0$.

Hence, if $f$ is separately convex in all variables $x_i$, you get that the diagonal of $A$ is non-negative. This, however, is not enough to guarantee the positive semi-definiteness.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.