0
$\begingroup$

I have following function $f(x_1,x_2\cdots x_K)$ $$f(x_1,x_2\cdots x_K)=\log\left(a+\sum_{i=1}^K\frac{b_i}{c+x_i}\right)$$ where $x_i$'s are all $\geq0$ and $a>0,b_i>0,>0$. I want to know how can we prove that the above function is jointly convex with respect to $(x_1,x_2\cdots x_K)$. I know we can use Hessian matrix to show this. So I have found following derivatives that are needed to construct the Hessian matrix $$\frac{\partial^2 f(x_1,x_2\cdots x_K)}{\partial x_i^2}=b_i(c+x_i)^{-3}\left(a+\sum_{i=1}^K\frac{b_i}{c+x_i}\right)^{-1}\left[2-b_i(c+x_i)^{-1}\left(a+\sum_{i=1}^K\frac{b_i}{c+x_i}\right)^{-1}\right]$$ and $$\frac{\partial^2 f(x_1,x_2\cdots x_K)}{\partial x_i \partial x_j}=-\frac{b_ib_j}{(c+x_i)^2(c+x_j)^2}\left(a+\sum_{i=1}^K\frac{b_i}{c+x_i}\right)^{-2}$$ Now if we can show that the resulting Hessian matrix is positive definite (or semi-definite) then we can say that the above function is convex. I do not know how to show that the resulting matrix has this property. Any help in this regard will be much appreciated. Thanks in advance.

$\endgroup$
  • $\begingroup$ I think you can prove the assertion by the definition of convex functions: $f(\lambda \vec x + (1-\lambda)\vec y) \cdots$ $\endgroup$ – Math Lover Feb 28 '18 at 4:19
  • $\begingroup$ @MathLover can you please add some steps? I will be very thankful to you. $\endgroup$ – Frank Moses Feb 28 '18 at 4:22
1
$\begingroup$

let $m=(m_1,m_2,m_3..m_K)$ and $n=(n_1,n_2,...,n_K)$ If the function is convex then $$d_1f(m)+d_2f(n)\ge f(d_1m+d_2n)$$where $d_1+d_2=1$ This means we need to prove $$\log\left(\left(a+\sum_{i=1}^K\frac{b_i}{c+d_1m_i}\right)^{d_1}\left(a+\sum_{i=1}^K\frac{b_i}{c+(1-d_1)n_i}\right)^{d_2}\right)\ge\log\left(a+\sum_{i=1}^K\frac{b_i}{c+ d_1m_i+d_2n_i}\right)$$

I am not sure how to prove this inequality but this should be easier than proving that hessian is positive semi definite.

$$\left(a+\sum_{i=1}^K\frac{b_i}{c+d_1m_i}\right)^{d_1}\left(a+\sum_{i=1}^K\frac{b_i}{c+d_2n_i}\right)^{d_2}\ge\left(a+\sum_{i=1}^K\frac{b_i}{c+ d_1m_i+d_2n_i}\right)$$

From Frank Moses' Comment $$\left(a+\sum_{i=1}^K\frac{b_i}{c+ d_1m_i+d_2n_i}\right)=\left(a+\sum_{i=1}^K\frac{b_i}{c+ d_1m_i+d_2n_i}\right)^{d_1}\left(a+\sum_{i=1}^K\frac{b_i}{c+ d_1m_i+d_2n_i}\right)^{1-d_1}$$ It is clear that $c+d_1m_i \le c+d_1m_i+d_2n_i$ using this we can prove the inequality given.

$\endgroup$
  • $\begingroup$ thank you for your answer. But I think your first equation corresponds to the $\theta=.5$ case only while I think it should be valid for all values of $0\leq \theta \leq 1$? What do you say? $\endgroup$ – Frank Moses Feb 28 '18 at 4:20
  • 1
    $\begingroup$ Yes you are right, I fixed that. $\endgroup$ – Piyush Divyanakar Feb 28 '18 at 4:25
  • $\begingroup$ The inequality you suggested is even simpler to prove Piyush. Just write $d_2=1-d_1$ and then use $\left(a+\sum_{i=1}^{K}\frac{b_i}{c+d_1m_i+(1-d_1)n_i}\right)=\left(a+\sum_{i=1}^{K}\frac{b_i}{c+d_1m_i+(1-d_1)n_i}\right)^{d_1}\left(a+\sum_{i=1}^{K}\frac{b_i}{c+d_1m_i+(1-d_1)n_i}\right)^{1-d_1}$. After this use the facts that almost every parameter in my question is non-negative. If this reasoning is wrong then please let me know. Thank you for your answer. $\endgroup$ – Frank Moses Feb 28 '18 at 4:51
  • 1
    $\begingroup$ Yes the reasoning is correct. I have included this in the answer. $\endgroup$ – Piyush Divyanakar Feb 28 '18 at 5:05
1
$\begingroup$

I believe that in this case the more efficient way to prove is via the composition rules. It is known that $$ g(y_0, y_1, \dots, y_n) = \log\left( \sum_{j=0}^n e^{y_j} \right) $$ is convex. It is the well-known "log-sum-exp" function, and proving that it is convex is quite easy. It is also increasing in each argument.

It is also easy to show using the second derivative that each of the functions $\log(b_j / (c + x_j))$ is also convex on $x_j \geq 0$ for $b_j,c>0$. The function $\log(a)$ is convex since it is a constant. Thus, the composition: $$ f(x_1, \dots, x_n) = g(\log(a), \log(b_1 / (c + x_1)), \dots, \log(b_n / (c + x_n)) = \log\left( a + \sum_{j=1}^n \frac{b_j}{c+x_j} \right) $$ is also convex by the theorem regarding the composition of a convex and increasing function over a convex function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.