Skip to main content

Questions tagged [optimal-control]

Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. (Def: http://en.m.wikipedia.org/wiki/Optimal_control)

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

Inconsistent solutions to linear optimal control problem

Consider the following optimal control problem: \begin{align} J(t) = \inf_{u(t)} \ & \frac{1}{2} \int_0^\infty e^{-\delta t} \left( x(t)^2 + \lambda y(t)^2 \right) dt \\ s.t. \ &u(t) \geq - \...
NC520's user avatar
  • 279
0 votes
0 answers
46 views

Find optimal control

I have the following problem: \textbf{Exercise 2.- A cup of coffee is initially at 100°C, and we want to lower its temperature to 0°C as quickly as possible by adding a fixed amount of milk. If $x(t)$ ...
Gonzalo de Ulloa's user avatar
1 vote
0 answers
10 views

Contraction of HJB operator

In Reinforcement Learning, contractivity of bellman operator plays an important role. I want to extend this result. I'm considering infinite horizon stochastic optimal control. Problem setting: $$ ...
出木杉英才's user avatar
0 votes
0 answers
7 views

Contractivity of HJB or Stochastic HJB operator

In Reinforcement Learning, contractivity of Bellman equation plays an important role. My question is HJB operator $T_1$ or Stochastic HJB operator $T_2$, which is the continuous time form of Bellman ...
出木杉英才's user avatar
0 votes
1 answer
79 views

How to analytically solve this PDE ? Or what is the ansatz? [closed]

$$\partial_t v - \alpha x \partial_x v+ x \partial_y v+ \frac{1}{2} \sigma^2\partial_{xx} v = 0$$ This PDE appears in HJB for linear quadratic gaussian control with cross terms of state and control.
Thiha Aung's user avatar
1 vote
1 answer
53 views

Adding time delay compensation to LQR

Is it as straightforward to add time delay compensation to LQR that uses full-state feedback, as it is with PID that uses error-based feedback? PID can use eg Smith predictor, but when I search for ...
J B's user avatar
  • 35
2 votes
1 answer
74 views

Optimal control and convexity

I consider an optimal control problem using the Bolza formulation : the state and control variable are respectively $x(t)\in X\subset\mathbb{R}^n$ $u(t)\in\mathbb{R}^m$ for $t\in[t_0,T]$ where $X$ and ...
G2MWF's user avatar
  • 1,339
0 votes
0 answers
28 views

How to solve Lagrange optimal control problem whose cost is independent of control?

I would like to solve an optimal control problem of the form $$ \inf_{u : [0, 1] \to \mathbb{R}_+} \int_0^1 L(x(t), t) \, dt \qquad \text{subject to} \qquad \dot{x}(t) = u(t) $$ where $L(x, t)$ is a ...
Max Daniels's user avatar
1 vote
0 answers
22 views

Can the optimal stopping time problem be handled by strong Markov property? [closed]

$X_t$ is a strong Markov process in $(\Omega, \mathcal{F},\mathcal{F}_t,\mathbb{P})$. $\tau$ is a stopping time, $T>0, \mathbb{E}_x(\cdot)=\mathbb{E}(\cdot|X_0=x)$. By Markov property, $\mathop{\rm{...
hua's user avatar
  • 11
0 votes
0 answers
16 views

General formulation of an optimal control problem

I am reading a paper on optimal control and the way they define the problem questionned me on the interest of such formulation. We consider $\Omega\subset\mathbb{R}^{n}$, $x(t)\in Y\subset\mathbb{R}^{...
G2MWF's user avatar
  • 1,339
2 votes
0 answers
48 views

Relationship discrete algebraic Riccati and discrete Lyapunov equation

Suppose that $K$ is the optimal control for an LQR problem with inputs $(A,B,Q,R)$, i.e.: $$ K = -(B^\top P B + R)^{-1} B^\top P A$$ where $P$ solves the discrete algebraic Riccati equation: $$P = A^\...
sdevlin's user avatar
  • 21
1 vote
0 answers
42 views

Solving backwards fractional differential equation with transversality condition

Let's take $\alpha\in(0,1)$ and $f,g\colon[0,T]\rightarrow \mathbb{R}$ good enough functions. When using the chain rule in fractional calculus in the context of fractional optimal control $$ \int_0^T ...
Aner's user avatar
  • 308
0 votes
0 answers
32 views

Optimal control problem by hand [duplicate]

I want to find the optimal control $u^*(t)$ for the following problem: \begin{align*} \dot{x}_1(t) &= x_2(t), & x_1(0) &= 3 \\ \dot{x}_2(t) &= -2x_1(t) + 5u(t), & x_2(0) &= 5 \...
lord voldemort's user avatar
3 votes
0 answers
119 views

When does an optimal input sequence for a discrete-time system exist?

Suppose an LTI discrete-time system is given by the equations $$ x_{k+1} = Ax_k + Bu_k,\\ y_{k} = Cx_k + Du_k $$ with $x_k\in\mathbb{R}^{m}$, $y_k\in\mathbb{R}^{n}$ and $u_k\in\mathbb{R}^{p}$ and $\...
Benjamin Tennyson's user avatar
1 vote
0 answers
23 views

Hamilton Jacobi Bellman and uniqueness

In the framework of optimal control we have introduced in class the HJB equation to solve optimal control problem of the form $$ \inf_{u\in\mathcal{U}}\left[\int_{0}^{T}L(t,x_u(t),u(t))dt + h(x_u(T));\...
G2MWF's user avatar
  • 1,339
0 votes
0 answers
29 views

truncating a system and plotting it on bode

Let system G(s) be: $$ G(s)=\sum_{i=0}^{10}\frac{(-1)^i}{(2i+1)^2}\frac{\omega_i}{s^2+2\zeta_i \omega_i s+\omega_i ^2}$$ $$\omega_i=\frac{(2i+1)\pi}{T}\, T=1\, \zeta_i=0.2$$ Its impulse response is an ...
Rice's user avatar
  • 19
0 votes
1 answer
97 views

Find Gain K and Time constant K of a system from the time response

There is a given system $\frac{K}{sT + 1}$ of order 1. The responses are in the image below and the 2 inputs are $u1(t) = 1(t)$ and $u_2(t) = \sqrt{2} \cdot \sin(\omega_2 t)$. How can I find the K and ...
sneha_jerin's user avatar
0 votes
0 answers
180 views

Optimal control problem with Hamiltonian linear in control

Let's consider the following deterministic optimal control problem, where $c(t)$ is the control, and $x(t)$ and $y(t)$ are the state variables: \begin{align} J(t) = \inf_{c(t)} \ &\int_0^\infty e^{...
NC520's user avatar
  • 279
0 votes
1 answer
30 views

Efficiency of constrained LQR formulation in CVXPY via batch-approach

I am interested in formulating a discrete finite time constrained LQR in CVXPY. \begin{align} \text{minimize } J = & \sum_{k=0}^N x'(k)Qx(k) + u'(k)Ru(k) \\\\ \text{subject to } & x(k+1) = Ax(...
Boyan Hristov's user avatar
0 votes
0 answers
56 views

Deterministic optimisation problem with inequality constraint

Let's consider the following deterministic constrained optimisation problem, where $c(t)$ is the control, and $x(t)$ and $y(t)$ are the state variables: \begin{align} J(t) = \inf_{c(t)} \ &\int_0^\...
NC520's user avatar
  • 279
1 vote
0 answers
42 views

Stability of the viscosity solution for the HJB - stochastic optimal control

I am following chapter 4 of this paper (1). Background information: The stochastic differential equations (eq’s 4.2 to 4.5) are given by: $$ \begin{aligned} & d S_t=\sqrt{\nu_t} d W_t \\ & d \...
THATS MY QUANT MY QUANTITATIVE's user avatar
0 votes
0 answers
41 views

Differentiability and continuity of the value function of optimal stopping problem

This question arise from Lemma 4.14 of Kwon, H. D., & Palczewski, J. (2022). Exit game with private information. arXiv preprint arXiv:2210.01610. Let us consider the following optimal stopping ...
Probvis's user avatar
  • 49
1 vote
1 answer
60 views

How can I prove this statement regarding a discrete-time algebraic Riccati inequality?

Consider the square matrix $X \in \mathbb R^{n \times n}$. In a paper that I'm currently reading, the authors state that, if the following two conditions are met: $$ \begin{align} &(1) \quad AXA^T ...
mhdadk's user avatar
  • 1,458
0 votes
1 answer
28 views

Finding the value of the parameter in integral that makes it zero (optimal control?)

I came across to the following question: let $f$ and $g$ be nice enough functions. I have $$I(\theta)\triangleq \int_a^b f(x)g(\theta x) dx.$$ Is it possible to find $\theta$ such that $I(\theta)=0$ ...
Martingalo's user avatar
  • 1,857
0 votes
1 answer
29 views

Under what condition is an inegral of a strong Markov process still (strong) Markov? [closed]

In Chapter iii section 6 of Optimal stopping and free boundary problems (Peskir and Shiryaev), they re-wrote the optimal stopping problem of the form $V=\sup_\tau\mathbb E(M(X_\tau)+\int^\tau_0 L(X_t)...
Probvis's user avatar
  • 49
0 votes
0 answers
39 views

How do we do dynamic optimization with a scalar constant as the only free variable?

Suppose we have a dynamic optimization problem, such as to minimize $$J(x_0,u) = \int_0^\infty \bigg(L(x,\dot{x}) +u^2 \bigg)dt$$ where $$\dot{x} = f(x) + g(x)u, \quad x_0 = x(t=0)$$ To be clear, $x$ ...
pawlyb's user avatar
  • 9
0 votes
1 answer
40 views

Ref Request: Infinite Dimensional Lagrange Multipliers, KKT Conditions, and Control

I'm looking for a more up-to-date book covering similar material as the second half of Luenberger's Optimization by Vectors Space Methods. That book covers Lagrange multiplier necessary conditions in ...
Smithey's user avatar
  • 695
1 vote
0 answers
24 views

Optimal control problem with inequality constraints at discrete times

I am interested in solving the following optimal control problem $$\min_{\mathbf{u}(t)} \ \phi(\mathbf{x}(t_f),t_f)+\int_{t_0}^{t_f}\mathcal{L}(\mathbf{x}(t), \mathbf{u}(t), t)\, \mathrm{d}t$$ ...
CaG's user avatar
  • 87
0 votes
0 answers
31 views

How can I apply the Hamiltonian function and Pontryagin's maximum principle in the context of Optimal Control Theory?

I am really struggling to grasp how the Hamiltonian Function and Pontryagin's Maximum Principle work in the context of Optimal Control Theory (Maths for Economics) course. I am given the following ...
astute-hoplite's user avatar
0 votes
1 answer
44 views

How can I get the inverse of the transfer function $\Phi$? a question from Page 56 of Lemma 4.5 in the book "ESSENTIALS OF ROBUST CONTROL" [closed]

Let $\Phi(s) = = \gamma^2 I + B^*\left( (sI-A)(C^*C)^{-1}(sI+A^*) \right)^{-1}B + B^*(sI+A^*)^{-1}C^*D - D^*C(sI-A)^{-1}B - D^*D.$ How to compute the inverse of $\Phi$, in Page 56 of Zhou's book "...
MatrixWong's user avatar
1 vote
0 answers
38 views

HJB equation with shape preserving constraints

I am trying to numerically solve some continuous-time macroeconomic models. For example, let's consider a simple growth model (as an HJB equation): $$ \rho V(k) = \max_{c} \frac{c^{1-\sigma}}{1-\sigma}...
TZh's user avatar
  • 11
0 votes
1 answer
94 views

How to solve discrete-time algebraic ricatti equation for affine dynamical system using scipy.linalg.solve_discrete_are

I am trying to solve a discrete algebraic ricatti equation for an affine dynamical system (ADS) of the form: $$x_{t+1} = A x_t + B u_t + b$$ where $x_t, u_t$ and $b$ are vectors and $A$ and $b$ are ...
Jabby's user avatar
  • 97
0 votes
1 answer
66 views

How can I define uniform convergence for this sequence of functions?

Consider the following recurrence relation $$ x_{k+1} = Ax_k + Bu_k $$ where $x_0 \in \mathbb R^n$ is the initial condition, $A \in \mathbb R^{n \times n}$ and $B \in \mathbb R^{n \times m}$, and $...
mhdadk's user avatar
  • 1,458
2 votes
0 answers
45 views

Source material to learn optimal control theory

I am interested to learn Optimal Control Theory from a computational point of view. I have taken courses like ODE/PDE, Calculus of Variation, Numerical Analysis, Optimization Methods, Scientific ...
Ajin Shaji Jose's user avatar
1 vote
1 answer
100 views

Are the $\inf$ and $\lim$ interchangeable in the infinite-horizon discrete-time LQR problem?

My question is about the infinite-horizon linear-quadratic regulator (LQR) problem. Consider the following linear and time-invariant system $$ x_{k+1} = Ax_k + Bu_k $$ Given the initial state vector $...
mhdadk's user avatar
  • 1,458
0 votes
0 answers
16 views

References to understand the foundations of optimal control

I am using optimal control for mathematical economics. However, many books by Kamien and Schwarz, Sydsaeter et Sierstad do not always enter into mathematical foundations of optimal control and give ...
optimal control's user avatar
0 votes
2 answers
63 views

Expected value of stopped iid sequence.

Let $X\ge 0$ a finite expectation continuous random variable with cdf $F$ and denote $\overline{F}=1-F$. Let $\{X_i\}$ a sequence of iid copies of $X$. Let $\tau_p$ be the following stopping rule: ...
xyz's user avatar
  • 990
2 votes
0 answers
39 views

How to design/add a new controller to a system without breaking the existing controller in the system? [closed]

Please help me to find related topics/books for this problem: For example, assume we have a water heater, and a tank of water. We can design a controller to heat the water in the tank and keep it in ...
Alex's user avatar
  • 31
0 votes
1 answer
116 views

Numerical gradient calculation for optimal control of PDE

I have two related questions regarding the correct way to perform numerical gradient descent for an optimal control problem with a non-uniform mesh/grid. 1) Say I am solving a PDE-constrained ...
Frubiclé's user avatar
  • 603
1 vote
0 answers
137 views

Optimal control: coarse control and fine dynamics

Consider the time evolution of a probability distribution $P(\alpha_{t})$ as follows. \begin{equation} P(\alpha_{t}) = F[P(\alpha_{t-1}), U_t(y_{t-1})], \end{equation} where $y_{t-1} = \mathcal{P}\...
Prakhar Godara's user avatar
3 votes
1 answer
88 views

Optimal Control and Dynamic Programming Principle

I am looking at the dynamic programming principle in optimal control problems. I am reading a book on the subject. This is the statement of the problem and approach. The books is "Non-Cooperative ...
Ramesh Kadambi's user avatar
1 vote
0 answers
58 views

Could someone recommend an excellent book on optimal control theory for partial differential equations?

I am a researcher in optimal control theory, particularly focusing on PDEs. I have extensively studied many of J-L Lions' excellent books in this field, as well as Fredi Tröltzsch's outstanding work. ...
MATAKA's user avatar
  • 23
0 votes
1 answer
65 views

Optimal control problem proving extremum

Let's consider the integral $F(x) = \int_{1}^{e}(x-t\dot{x})dt$, with initial conditions $x(1) = 1, x(e) = 2$. The problem is to find extremum of the integral above. My approach: I was able to find ...
wxist's user avatar
  • 481
0 votes
0 answers
52 views

What is the Frechet derivative of $\cos(x(\cdot))$?

What is the Frechet derivative of $\cos(x(\cdot))$? Let $f: C[0,1] \rightarrow \mathbb{R}$, $f(x(\cdot))$. My approach: by the definition of Frechet derivative we need to consider $$ f((x+h)(t)) - f(...
wxist's user avatar
  • 481
0 votes
0 answers
39 views

Cost/Objective Function Normalization (MPC)

I am trying to develop an MPC. In this MPC, I predict the temperature and try to bring the sensor value to the desired setpoint temperature. I predict the temperature in the next 180 minutes for the ...
Clankk's user avatar
  • 1
0 votes
1 answer
66 views

Defining white noise intensites in state space kalman filter with spectral factorization

I'm a noob on this subject, so please be extra clear :) With a system of equations: $$\dot\omega_x = \alpha_s\omega_y - \epsilon\omega_s\omega_y + \epsilon\omega_s\eta + Q_x$$ $$\dot\omega_y = -\frac{\...
Zacharias Andersson's user avatar
0 votes
1 answer
65 views

Riccati equation in matlabs icare solver

I'm using the icare riccati equation solver from MATLAB to solve lqr problems were the constraint is given by the equation $My^{\prime} = Ay+Bu$ where for context M (mass matrix), A (stiffness matrix) ...
mlg3371333's user avatar
1 vote
2 answers
83 views

Stochastic control problems in infinite horizon

Consider the following maximization problem and the wealth dynamics $$\max \mathbb{E}\left[\int_0^{\infty} \frac{1}{\gamma} e^{-\beta t} c_t^\gamma \mathrm{d} t\right]$$ $$\mathrm{d} X_t=X_t\left(r+\...
mnmn1993's user avatar
  • 425
2 votes
0 answers
56 views

Is it possible to derive, if informally, the Hamilton-Jacobi-Bellman equation from Feynman-Kac?

HJB Suppose you have a stochastic process $\left( X^u_t \right)_{t \in[0,T]}$ controlled by $u(t, x)$. We can define the optimal value function $$ V(t, x) = \min_u \mathbb{E} \left[ \int_t^T C(s, X_s, ...
user357269's user avatar
0 votes
0 answers
10 views

How to add a linear contraint between state variables to a current time Hamiltonian?

Let's say I have an objective function $F$ with state variables $A,B,C$ and relative equations of motion, I can create the current time Hamiltonian with $H_C = F\{A,B,C\}+\alpha * \dot A+\beta * \dot ...
Antonello's user avatar
  • 261

1
2 3 4 5
21