Questions tagged [optimal-control]

Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. (Def: http://en.m.wikipedia.org/wiki/Optimal_control)

Filter by
Sorted by
Tagged with
1 vote
2 answers
39 views

How to solve second order differential optimal control or optimization problem?

From a long time, I meet an optimal control problem, but i don't know how to solve it. Well, to be more specific, Suppose we have following dynamic system and cost function, \begin{cases} \ddot{y}=\...
user avatar
1 vote
0 answers
21 views

Lagrange multipliers: discrepancy between optimization and adjoint sensitivity results

Consider the constrained optimization problem: $min_x f(x)$ s.t. $g(x)=0$. For simplicity, let $f$ and $g$ be scalar functions. Under suitable conditions, the Lagrange multiplier theorem gives: $\...
user avatar
  • 63
2 votes
1 answer
20 views

LQR definitions

I have to define the choice of parameters I have chosen to create an LQR controller for a drone, and I have written the following: High penalties in the Q matrix mean that the state will try to ...
user avatar
1 vote
1 answer
38 views

Does "Iterative Policy Evaluation" Guarantee the "Optimal Policy"?

I am reading about "Iterative Policy Evaluation" algorithm in the context of Reinforcement Learning (http://incompleteideas.net/book/ebook/node41.html). If I have understood this correctly - ...
user avatar
1 vote
0 answers
12 views

Pontryagin Maximum Principle with terminal and initial conditions

Consider a control problem with Lagragian $L(t,x,u)$ (where $u$ is the control, $x \in \mathbb{R}^d$ the state) and dynamics $\dot{x}=f(x,u,t)$. I have mostly seen problems in which the dynamical ...
user avatar
  • 41
0 votes
0 answers
28 views

In optimum control, what is intended by discrete scaling?

I have been looking for strategies to solve my optimal control problem as it is severely suffering from scaling issues. I then came across this article (https://arxiv.org/pdf/1810.11073.pdf) and it ...
user avatar
-2 votes
0 answers
76 views

Feedback linearization with integral action - How?

Assume that you know sort of the dynamics of the system. It's not 100% perfect, but it's at least 90% perfect. $$\dot x = f(x, u)$$ I want to find a control law that suits this system. I have been ...
user avatar
  • 2,497
1 vote
1 answer
41 views

A matrix $A$ is Hurwitz, $e^{Ah}$ is Schur. Discretization of continuous-time linear stochastic system and its stability.

Suppose we have a real-valued square matrix $A$ that is Hurwitz, i.e., all eigenvalues of $A$ have strictly negative real parts. I want to show that given a scalar $h>0$, if A is Hurtiwz, then $e^{...
user avatar
0 votes
0 answers
28 views

proof of the existence of a bang bang control

I was reading the proof of the existence of the bang bang control throughout the theory of the L infinty space; however i got stuck at a piece of the proof. I am sure im overcomplicating the problem ...
user avatar
0 votes
0 answers
48 views

Calculating the limit of a Wasserstein distance of two SDE's

I am trying to prove that: $\lim_{t \to \infty} W_2(\mu_t, \nu_t) = 0 $ where we have that $\mu_t = Law(X_t)$ and $\nu_t = Law(Z_t)$ with $$dX_t = -h(X_t)dt + \sqrt(\frac{2}{\beta})dB_t$$ $$dZ_t = -h(...
user avatar
0 votes
0 answers
65 views

optimal spread modeling

Here is an interesting case. A bacterium either doubles or transforms into an infectious form with a time-dependent probability $p_n$ and $1-p_n$. Let $X_n$ be the number of duplicate bacteria and $...
user avatar
0 votes
0 answers
21 views

Continuity of Parabolic Evolution Equation

Let $\Omega\subset \mathbb{R}^d$ be a compact set with smooth boundary and $V$ be a (strongly) convex, smooth function. For the equation $$ \partial_t\rho_t = \nabla\cdot(\rho_t\nabla(\log\rho_t + V)) ...
user avatar
0 votes
1 answer
68 views

Solution to a second order semilinear PDE, linear in all derivatives.

I am trying to prove global existence and uniqueness, or even better, find an analytical solution, or at least some form to get rid of one of the dimensions to the following semilinear PDE $$ 0=u_{t}-...
user avatar
  • 23
0 votes
1 answer
89 views

How to carry out the expected value of the cost function in a LQG problem to tackle path tracking?

I have a system, whose state is defined by $x_t$. The transition state mapping (STP) for the systems is defined as: $$x_{t+1} = A_t x_t + B_t u_t + w_t$$ where, $x \in \mathbb{R}^{n \times 1}$, $A_t \...
user avatar
  • 147
1 vote
0 answers
29 views

Non-linear differential equation existence and uniqueness of solution

I am very unfamiliar with the theory of nonlinear differential equations. Is there any result that given some initial conditions ensure the existence and uniqueness of a solution to the following: $af(...
user avatar
2 votes
0 answers
42 views

Stochastic Optimal Control

The general formulation of a stochastic control problem with finite horizon is $$V(x)=\sup_{u\in\mathbb{U}}E\left(G(X_{T}^{u}) + \int_{0}^{T}F(s,X_{s}^{u}, u_{s})ds|X_{0}^{u}=c\right)$$ where $X_{s}^{...
user avatar
1 vote
0 answers
28 views

Optimal Reinsurance for a diffusion approximation

What is the relation between the solutions of the optimal reinsurance problem for a diffusion approximation and the optimal reinsurance problem for the classical risk model? Apparently this result in ...
user avatar
1 vote
0 answers
38 views

Regarding an equation in an iteration method

I am going through a paper on an iterative method related to discrete optimal regulator, G. Hewer, "An iterative technique for the computation of the steady state gains for the discrete optimal ...
user avatar
  • 113
1 vote
1 answer
50 views

show that the nullspace of $P$ is invariant when applying state matrix

Assume $P$ solves the Riccati equation $${A^T}P + PA - PB{B^T}P + {C^T}C = 0$$ show that nullspace($P$) is invariant up to the application of the state matrix and show that $kernel(P) \subset C$ ...
user avatar
  • 35
2 votes
0 answers
24 views

Optimal stopping problem with restricted stopping times

Consider that you have some ergodic, time-homogenous Markov process $(X_n)_{n \ge 0}$ taking values in a finite space $S$. To solve $$ g(N, x) = \sup_{\tau \le N} E \left[ f(X_\tau) \mid X_0 = x \...
user avatar
  • 516
0 votes
0 answers
25 views

How to solve this problem with calculus of variation?

Let $c \in (0,1)$ and $n\geq 2$ be some integer. Suppose we can choose twice differentiable function $g: [0,1]\to R$ to solve the following inequality constrained program $$max_g \int_{c }^1 x g(x)dx$...
user avatar
  • 97
0 votes
0 answers
12 views

Dependence of V in ergodic infinite time optimal control problem on cost function

Consider the ergodic, infinite time optimal control problem: dx = [F(x) + G1 u]dt + G2 dW J = lim T->infinity E{ 1/T\int_0^T [Q(x) + u'Ru]dt}, F(0) = 0, Q(0) =0, Q(x) >= 0 Now suppose that Q(x) ...
user avatar
  • 1
0 votes
1 answer
56 views

Creating a MATLAB program using Euler Explicit Method [closed]

I want to solve following ODE using Euler explicit method. $$S'(t)=\frac{b}{m}\bigg(u(t)+\frac{P(t)}{p}\bigg)$$ where $S(0)=\frac{q}{m}$. We can have any value for the parameters. $P(t)$ should be a ...
user avatar
6 votes
1 answer
86 views

Optimal speed for approaching red light to maximize velocity with non-uniform probability

Problem statement When I cross red lights, my goal is to being going as fast as possible when the light turns green. I am at distance $D$ from a traffic light when it turns red. Let the time length of ...
user avatar
  • 135
0 votes
1 answer
30 views

State constraint on stochastic optimal control

I have been reading "Partial Differential Equation Models in Macroeconomics" (Achdou, 2014). One of the problems $$\max_{\{c_{t}\}} \mathbb{E}_{0} \int_0^{\infty} e^{-\rho t} u(c_t)\mathrm{d}...
user avatar
  • 296
1 vote
0 answers
41 views

Compactness in Kantorovich Duality Problem

I've been following along in https://lchizat.github.io/files2020ot/lecture1.pdf to learn about optimal transport theory and ran into some confusion in Chapter 4 "The dual problem"... Let $X,...
user avatar
  • 5,481
0 votes
0 answers
44 views

PID controller without transfer function of plant

I have a basic question about PID controllers. According to many of books, plant model should be formed in s-domain (transfer function) to implement PID control procedure to keep the reference signal. ...
user avatar
  • 39
0 votes
0 answers
34 views

Modelling an S-shaped Second Order Response

I am trying to model an S-shaped second order process response as a transfer function in Simulink, as shown in Figure 3 in the linked article. Ideally the response should be overdamped. I have tried ...
user avatar
1 vote
1 answer
114 views

Solving stochastic control problems using Hitsuda representation

I would like to solve the following problem. Consider a financial market with quadratic transaction costs, one risky asset with price dynamics: $S_t = s_0 + \mu t + \sigma W_t$, for $t \geq 0 , \...
user avatar
1 vote
1 answer
68 views

Any stabilizing control law $K$ is optimal for some LQR problems ($Q$ and $R$).

I recently went through Kalman's paper "When is a Linear Control System Optimal" published in 1964. The paper makes me wonder whether the following statement is true: Any stabilizing control ...
user avatar
1 vote
0 answers
18 views

Derivatives of a 2D B-spline with respect to the control points

I'm dealing with an optimal control problem and I want to solve it with the B-splines. In order to compute the gradient of my objective function, I have to derive the B-spline with respect to the ...
user avatar
0 votes
2 answers
78 views

Showing that $x^*(t)=\sin(t)$ minimizes the functional $J(x(t)) = \int_0^{\pi/2} [\dot x(t)^2 - x(t)^2 ]dt$

We are given the functional $$J(x(t)) = \int_0^{\pi/2} [\dot x(t)^2 - x(t)^2 ]dt$$ with the fixed boundary condition $x(0)=0$ and $x(\frac{\pi}{2})=1$. Could anyone help me prove that $x^*(t)=\sin(t)$ ...
user avatar
  • 21
1 vote
1 answer
72 views

Why do optimal control and reinforcement learning use different notation?

In optimal control, state is $x$, control is $u$, and dynamics are $\dot{x}=f(x,u)$. In reinforcement learning, state is $s$, action is $a$, and dynamics are $s'\sim P(s'|s,a)$. I'm curious why these ...
user avatar
1 vote
0 answers
16 views

Functional Extremum Problem with inhomgeneous boundary condition

How to solve a functional extremum problem where the boundary condition is inhomogeneous? let's say $$\begin{aligned}&\max_{F}\int_p^11-F(v)dv\\ s.t.& 1-F(p)-pF'(p)=0\\ &F(p) \text{ is a ...
user avatar
  • 43
0 votes
2 answers
64 views

Bound the norm of a matrix function related to discrete algebraic Riccati equation

I was going through the following paper on perturbation analysis of the discrete Riccati equation. https://dml.cz/bitstream/handle/10338.dmlcz/124552/Kybernetika_29-1993-1_2.pdf. The perturbation ...
user avatar
2 votes
1 answer
53 views

Pontryagin principle for a system $\dot{x}=M(x)u$ and quadratic cost

Context: I am studying systems of the form $\dot{x} = M(x)u$ for some state dependent matrix $M(x)\in\mathbb{R}^{n\times m}$ with $m<n$, $x\in\mathbb{R}^n, u\in\mathbb{R}^m$ and initial condition $...
user avatar
5 votes
0 answers
79 views

Optimal control of posterior belief over a finite horizon

$\large \textbf{Preface:}\ $ Below I describe a dynamic programming problem I am not sure how to formalize. In short: a (Bayesian-updating) agent sequentially runs costly experiments over a finite ...
user avatar
0 votes
0 answers
23 views

Multi-Stage Combinatorial Optimization

I am not sure if I used the corret terminology. I think the problem is a multi-stage combinatorial optimization problem. The problem is like this: There is a dynamic equation $$ S_{k+1} = f(S_k,\pi_k) ...
user avatar
2 votes
1 answer
61 views

Determining the number of switching times in a bang-bang control with nonlinear switching function

In optimal control, if the Hamiltonian $H$ is linear in the control $u$, then the optimality condition $$\frac{\partial H}{\partial u}$$ gives no information about the optimal control $u^*$. The way ...
user avatar
  • 479
2 votes
0 answers
21 views

Sufficient condition for convex conjugates (does one imply the other?)

We say $(f_1,f_2,\cdots,f_N)$ a convex conjugate if for any $i=1,2,\cdots,N$ and any $x_i\in\Bbb R^d$, we have: $$f_i(x_i)=\sup\left\{\sum_{k=1}^{N}\sum_{j=k+1}^N x_k x_j - \sum_{j=1,j\neq i}^N f_j(...
user avatar
1 vote
1 answer
83 views

shortest distance between points avoiding eclipse

What is the shortest path between two points (-3,0) and (3,0) that avoids the interior of the ellipse $\frac{x^2}{2}+\frac{y^2}{1}=1$ I should use the maximum principle for state constraints, but I ...
user avatar
  • 317
0 votes
0 answers
57 views

Optimal control of an integrator

Given the control problem $$ \max_{u \in [0,1]} \int_0^{10} x (t) \,{\rm d} t \quad \text{subject to} \quad \dot x = u, \quad x(0)=0, \quad x(10)=2$$ a) Find the solution $(x_∗,u_∗)$ that satisfies ...
user avatar
0 votes
0 answers
21 views

Properties of the value function of a stochastic control problem

I want to find the properties of the value function in a stochastic control problem. Consider the following problem: \begin{align} V(x_1,x_2) = &\inf_{u_1,u_2} E\left[\int_0^\infty C_1 \max(0,X_1(...
user avatar
  • 13
0 votes
0 answers
27 views

Existence of visous solution to Hamilton-Jacobi equation for convex Hamiltonians

I have a conceptual misunderstanding about Chapter 10 of Evans' book on PDEs. It seems the author wants to show that for convex Hamiltonians, the Hamilton-Jacobi equation $u_t+H(Du,x)=0$ has a viscous ...
user avatar
  • 3,848
0 votes
0 answers
31 views

what's dual of this cone optimization problem, Is it a linear problem?

I want to know the dual of this problem: \begin{equation} \begin{aligned} &\min_{x,t}f^Tx\\ s.t.&\|A_ix+b_i\|\le c_i^Tx+d_i\quad i=1,\dots,n\\ &\begin{bmatrix} C_0+\sum_{i=1}^nx_iC_i&...
user avatar
1 vote
1 answer
42 views

Converting a nonlinear model predictive problem to parametric optimization problem

It is very well known that a linear model predictive control problem \begin{align} \label{eq:linear-original problem} \begin{aligned} &\text{minimize}_{(u_{t})_{t=0}^{N-...
user avatar
0 votes
0 answers
42 views

Formulating the minimum distance between two points as an optimal control problem.

I came across this link that formulates the brachistochrone problem as an optimal control problem: brachistochrone. Instead of minimizing the time, I'd like - by using the same dynamics - to make the ...
user avatar
  • 1
1 vote
0 answers
52 views

Optimal control problem with boundaries depending on control

I have a relatively standard optimal quadratic control problem on infinite horizon : $\int_0^\infty (R-u)^\top (R-u) + C(t)^\top u~ dt $ subject to $\dot R = kR - k u $ but with one specific. The ...
user avatar
1 vote
1 answer
27 views

H2-filtering of unstable LTI systems. Can this problem be reformulated as convex optimization problem with LMI constraints?

Consider the discrete-time generalized LTI plant with minimal state-space realization $$x_{k+1}=A_d x_k + B_{d1}w_k\\z_k=C_{d1}x_k+D_{d11}w_k\\y_k=C_{d2}x_k+D_{d21}w_k$$ For the Schur-stable $A_d$ ...
user avatar
3 votes
1 answer
79 views

Optimal stopping of a Poisson Process with a risky reward

I'm confident that there is a well-known solution to this problem, but I am having trouble finding a reference for it. I am also quite rusty on these kinds of problems, so I am having trouble solving ...
user avatar

1
2 3 4 5
18