Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am currently trying to study the Newton's method of optimization through this wiki article http://en.wikipedia.org/wiki/Newton%27s_method_in_optimization. However, I didn't get this concept about constructing the sequence xn and approximating the objective function by a quadratic function around xn.

Can anyone provide me some good references. I mean why are are constructing that sequence xn? Any geometric visualization will be helpful I guess.

share|improve this question

migrated from stats.stackexchange.com Jun 15 '12 at 21:42

This question came from our site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.

add comment

1 Answer 1

This is not really very difficult. The article gives the second order Taylor expansions of a twice differentiable function f. This function is quadratic in ∆x. A quadratic function will have a single extreme point where the first derivative is zero. It is either a maximum or a minimum depending on whether the coefficient of the quadratic term is positive or negative. If you start at a point xn that is reasonably close the the target point x the second order Taylor series will be a good local approximation for the function. So you set the first derivative of that function of ∆x to 0. That gives ∆x =-f'(xn)/f"(xn) and hence x-xn=-f'(xn)/f"(xn) since x-xn=∆x. So we have that to get closer to x we take xn+1 =xn-f'(xn)/f"(xn). Now expanding at the point xn+1 gives the next iteration xn+2=xn+1-f'(xn+1)/f"(xn+1).

share|improve this answer
    
can you elaborate more when you say If you start at a point xn that is reasonably close the the target point x the second order Taylor series will be a good local approximation for the function. –  rajan sthapit Jun 16 '12 at 12:24
    
Reasonably close just means close enough that the second order Taylor series approximation of the function at that point is close to the actual value of the function at that point. To use this formula you need to be starting close enough that the formula will move you in the right direction toward x. So xn needs to be close enough that the move to xn+1 gets you closer to x. This is admittedly a little vague but you can pick an epsilon to make this precise. –  Michael Chernick Jun 16 '12 at 14:36
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.