# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

475 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
97 views

### Closed form solution to the Digamma recurrence relation?

The difference between two digamma function can be written using the following recurrence relation: $\psi(n+z) - \psi(z) = \sum_{i=0}^{n} \frac{1}{i + z}$ My question is, is there a closed form ...
37 views

### Expectation of Value Function

I am not sure whether this is the right place to ask this. I have solved a standard Bellman equation problem. The Value Function $V$ depends on 3 state variables: $K_t$, $X_t$, $Z_t$. The variables ...
319 views

### Why does Frobenius norm make BFGS scale-invariant?

On slide 11 here it is claimed that the weighted Frobenius norm leads to a scale-invariant optimization method. Similar claims about this norm can be found throughout the literature see 1,2,3. In ...
467 views

### Steepest descent method - proof in Nocedal and Wright

In Numerical Optimization by Nocedal and Wright, Chapter 2 on Unconstrained Optimization, (http://home.agh.edu.pl/~pba/pdfdoc/Numerical_Optimization.pdf) beginning on page 20, they verify that the ...
73 views

### Effect of Scale of Data and Objective Function in the Convergence of Gradient Descent

One way to tune step size in gradient descent is via backtracking line search. backtracking line search (with parameters α ∈ (0, 1/2), β ∈ (0, 1)) starting at $t = 1$, repeat $t := \beta t$ ...
28 views

### Numerical Method for fitting parameters of an explicit integration to actual data

I have a heat transfer system described by, $$\{\dot{T}\} = [C^{-1}]\left([K]\{T\} + \{F\} \right)$$ where ${T}$ is a vector of the nodal temperatures of the system. From initial conditions I am able ...
84 views

### Can someone help me understand using the Jacobian matrix with Newton's Method for finding zeros?

I'm struggling to understand approximating solutions to non linear equations using a Jacobian matrix. I understand intermediate steps, but I'm unsure how everything comes together. I want to use ...
335 views

### Computational complexity of conjugate gradient method for a positive semidefinite Hermitian matrix

Let us assume that we want to solve the linear system: $$\mathbf{A}\mathbf{x} = \mathbf{b}$$ with the conjugate gradient method. $\mathbf{A}$ is a positive semi-definite Hermitian matrix. The ...
51 views

### Calculating block diagonalization / canonical bases with linear optimization?

Edit Even though I have started answering my own question I am still eager to hear any feedback and new ideas. So feel free to tell me if you come to think of anything. In Linear Algebra there are ...
61 views

### Find the “optimal” placement of $n$ points in a polygon

I was hoping someone could help me with a question I've been pondering the past few days. I searched the usual places (google, journals, texts, etc) but couldn't find anything that fit the bill, but ...
51 views

### Numerical stability of computational results

Let z be a function of a finite number of variables i.e. z=f(a,b,c,...). If we have the mathematical formula connecting z and the variables, we can determine how the value of z varies with a change ...
681 views

90 views

### Maximization of a nasty Gaussian likelihood

I have a Gaussian likelihood function, $$p(y|x) = \mathcal{N}(y; Ax, (x^\top V x + \lambda) \otimes I)$$ where $A,V,\lambda$ is known, and $\otimes$ is the Kronecker product. (the notation indicates ...
119 views

38 views

276 views

### Linear Programming, Optimal Solutions

I posted the whole question to give some context, but my problem lies with (iv). I think you're meant to use a formula for the generalization of the optimal solution, but I'm not really sure what this ...
242 views

### using lsqcurvefit to fit piece-wise linear

I would like to use this function to fit piece-wise linearly to a set of data. Namely, I want to fit them with several linear segments. Including other requirements, I would not want the segments ...
51 views

### How to know all errors in Optimization process?

I am using fminsearch in MATLAB to iteratively find the answer of a function. But it seems like the fminsearch cannot fulfill my expectation. So I ask this question whether there is any way to do what ...