What is the advantage of adding $\log$ Barrier to solve a Linear program?

Let $$A \in \mathbb{R}^{n \times m}$$, $$b \in \mathbb{R}^{n}$$, and $$x \in \mathbb{R}^{m}$$. Let $$Ax \leq b$$ be the set of linear inequalities which can be written as $$a_i^Tx-b_i \leq 0 \,\,\,\,\forall i= 1, \ldots , m$$. Consider the following linear program

$$\min_{Ax\ \leq\ b} c^Tx \tag{1}$$ where $$c \in \mathbb{R}^{m}$$ is an arbitrary vector. If this problem has a solution, then according to representation theorem (or Weyl-Minkowski) the solution happens at vertices of the constraint set. Convex optimization on page 499 says $$f(x)=-\sum_{i=1}^m \log (b_i -a_i^Tx)$$ is a self-concordant $$\log$$ barrier function for linear inequalities. So, the above constraint problem can be an unconstraint problem as $$\min_{x \in \mathbb{R}^{m}} c^Tx -\sum_{i=1}^m \log (b_i -a_i^Tx) \tag{2}$$

Let the solutions of (1) and (2) be $$x_1^\ast$$ and $$x_2^\ast$$ respectively.

1- Are the solutions the same ($$x_1^\ast=x_2^\ast$$)? (I mean, Barrier bans the algorithm to approach to the boundary of the constraint set because it goes to infinity. However, the solution is on the boundary. How can (2) catch the solution?)
2- In terms of iterations and algorithm complexity which one is superior? I know for (2) we can use Newton's method to find the answer but how using Newton's method accelerate the algorithm?

3- Can we analytically show that the two problems are the same or one has superiority to the other?

• Ah, thanks, yes! – jjjjjj Jan 17 '19 at 0:02

1. No, the solutions are not the same. On p. 499, the book is merely presenting an example of a self-concordant function.

2. When you ask about algorithm complexity for problem (1), which algorithm do you have in mind? At this point in the book's development, it is not clear how to solve problem (1) efficiently. The book will present a strategy that utilizes log-barrier functions in chapter 11.

3. No, the problems are not equivalent.

You should read chapter 11 (Interior-point methods), and specifically section 11.2, which presents a strategy to minimize $$f_0(x)$$ subject to the constraints $$f_i(x) \leq 0, Ax = b$$ by solving a sequence of subproblems of the form \begin{align} \text{minimize} & \quad f_0(x) + \sum_{i=1}^m - (1/t) \log(-f_i(x)) \\ \text{subject to} &\quad Ax = b \end{align} with increasing values of $$t$$. Newton's method is used to solve each subproblem. As $$t$$ increases, the solution of the subproblem becomes a better approximation to the solution of the original problem. That is how log-barrier functions are used to solve problems with inequality constraints.

• I appreciate for your patience but I am little bit confused. Linear program is a convex problem so it can be solved using interior point method? Is it true? I have the following questions to understand my question better. Are these solutions close, i.e. $x_1^{\ast}$ and $x_2^{\ast}$? Can (2) be seen as the subproblem of (1) which gives an approximate solution? How many subproblem we need to define to solve (1) using interior point method? What is the optimal value of $t$ for getting a close solution to $x_1^{\ast}$? – Saeed Jan 17 '19 at 0:21
• Yes, a linear program is a convex problem, and a linear program can be solved using an interior point method. In fact, interior point methods often give state of the art performance for solving LPs. Interior point methods were originally developed for linear programming problems, and only later were extended to more general convex problems. – littleO Jan 17 '19 at 0:26
• No, the solution to problem (2) is not necessarily very close to the solution to problem 1. However, if you had written problem (2) as minimizing $c^T x - (1/t) \sum_{I=1}^m \log(b_i - a_i^T x)$, then we could say that if $t$ is large then the solution to problem (2) is close to the solution to problem (1). – littleO Jan 17 '19 at 0:29
• When the factor of $1/t$ is included, and $t$ is large, then the solution to (2) gives a good approximation to the solution of (1). Regarding how many subproblems you need to solve, and how large $t$ needs to be, I would say just read chapter 11 for more info about that. – littleO Jan 17 '19 at 0:31
• Thank you so much. – Saeed Jan 17 '19 at 1:02