# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

475 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0answers
97 views

### Closed form solution to the Digamma recurrence relation?

The difference between two digamma function can be written using the following recurrence relation: $\psi(n+z) - \psi(z) = \sum_{i=0}^{n} \frac{1}{i + z}$ My question is, is there a closed form ...
0answers
37 views

### Expectation of Value Function

I am not sure whether this is the right place to ask this. I have solved a standard Bellman equation problem. The Value Function $V$ depends on 3 state variables: $K_t$, $X_t$, $Z_t$. The variables ...
1answer
319 views

### Why does Frobenius norm make BFGS scale-invariant?

On slide 11 here it is claimed that the weighted Frobenius norm leads to a scale-invariant optimization method. Similar claims about this norm can be found throughout the literature see 1,2,3. In ...
0answers
467 views

### Steepest descent method - proof in Nocedal and Wright

In Numerical Optimization by Nocedal and Wright, Chapter 2 on Unconstrained Optimization, (http://home.agh.edu.pl/~pba/pdfdoc/Numerical_Optimization.pdf) beginning on page 20, they verify that the ...
0answers
73 views

### Effect of Scale of Data and Objective Function in the Convergence of Gradient Descent

One way to tune step size in gradient descent is via backtracking line search. backtracking line search (with parameters α ∈ (0, 1/2), β ∈ (0, 1)) starting at $t = 1$, repeat $t := \beta t$ ...
0answers
28 views

### Numerical Method for fitting parameters of an explicit integration to actual data

I have a heat transfer system described by, $$\{\dot{T}\} = [C^{-1}]\left([K]\{T\} + \{F\} \right)$$ where ${T}$ is a vector of the nodal temperatures of the system. From initial conditions I am able ...
1answer
84 views

### Can someone help me understand using the Jacobian matrix with Newton's Method for finding zeros?

I'm struggling to understand approximating solutions to non linear equations using a Jacobian matrix. I understand intermediate steps, but I'm unsure how everything comes together. I want to use ...
0answers
335 views

### Computational complexity of conjugate gradient method for a positive semidefinite Hermitian matrix

Let us assume that we want to solve the linear system: $$\mathbf{A}\mathbf{x} = \mathbf{b}$$ with the conjugate gradient method. $\mathbf{A}$ is a positive semi-definite Hermitian matrix. The ...
1answer
51 views

### Calculating block diagonalization / canonical bases with linear optimization?

Edit Even though I have started answering my own question I am still eager to hear any feedback and new ideas. So feel free to tell me if you come to think of anything. In Linear Algebra there are ...
0answers
61 views

### Find the “optimal” placement of $n$ points in a polygon

I was hoping someone could help me with a question I've been pondering the past few days. I searched the usual places (google, journals, texts, etc) but couldn't find anything that fit the bill, but ...
0answers
51 views

### Numerical stability of computational results

Let z be a function of a finite number of variables i.e. z=f(a,b,c,...). If we have the mathematical formula connecting z and the variables, we can determine how the value of z varies with a change ...
0answers
681 views

1answer
90 views

### Maximization of a nasty Gaussian likelihood

I have a Gaussian likelihood function, $$p(y|x) = \mathcal{N}(y; Ax, (x^\top V x + \lambda) \otimes I)$$ where $A,V,\lambda$ is known, and $\otimes$ is the Kronecker product. (the notation indicates ...
0answers
119 views

1answer
38 views

1answer
276 views

### Linear Programming, Optimal Solutions

I posted the whole question to give some context, but my problem lies with (iv). I think you're meant to use a formula for the generalization of the optimal solution, but I'm not really sure what this ...
0answers
242 views

### using lsqcurvefit to fit piece-wise linear

I would like to use this function to fit piece-wise linearly to a set of data. Namely, I want to fit them with several linear segments. Including other requirements, I would not want the segments ...
1answer
51 views

### How to know all errors in Optimization process?

I am using fminsearch in MATLAB to iteratively find the answer of a function. But it seems like the fminsearch cannot fulfill my expectation. So I ask this question whether there is any way to do what ...
0answers
70 views

### Effect of approximating a non-differentiable function on optimisation of minimisation

I am looking at a problem of constrained minimization, where the function to be minimized contains the Heaviside function, and as such is not twice continuously differentiable. My question is what ...
0answers
160 views

### bad convergence of steepest descent

Under which circumstances does the steepest descent method converge badly? I know if the search direction is approximately perpendicular to the descent direction the steepest descent method converges ...