Gradient Descent: Cost Function

Question

I'm trying to implement the gradient descent method for the problem of minimising the following function:

$$f(x) = \frac{1}{2}(x-m)^{T}A(x-m)-\sum\limits_{i=1}^n\log\left(x_i^{2}\right),$$

where $x \in R^n$ is a vector; $m \in R^n$ is a fixed vector; and $A$ is a fixed positive definite matrix.

The only applications of gradient descent I have come across is for linear regression! So, as a starting point for helping me to solve this, I'd like to know in what situations this cost function would be applied. Does anyone out there recognise it?

Answer 1

From the question, we have that f is a convex quadratic function

$$f(x) = \frac{1}{2}(x-m)^{\mathrm T} \mathrm A (x-m)-\sum\limits_{i=1}^n\log\left(x_i^{2}\right),$$

where $x \in R^n$ is a vector; $m \in R^n$ is a fixed vector, and $A$ is a fixed positive definite matrix (symmetric and positive semidefinite).

$$\nabla f ( x) = \frac{\mathrm T+1}{2} \mathrm A (x-m)^{\mathrm T}-\sum\limits_{i=1}^n\left(\frac{2x}{x_i^{2}ln(10)}\right)$$

Using gradient descent with step $\mu$,

$$ x_{k+1} = x_k - \mu \nabla f ( x_k)$$

Choose $\mu$ such that $f(x_k) < f(x_{k+1}) $,then do a loop until we find $x^{*}: f(x^{*}_k) - f(x^{*}_{k+1}) \sim 0$

This is a general way to take gradient descent for a convex quadratic function in n-dimensional space. Hope it is helpful.