Questions tagged [optimal-control]
Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. (Def: http://en.m.wikipedia.org/wiki/Optimal_control)
950
questions
1
vote
1
answer
75
views
How can I approach the target temperature as fast as possible given the transfer function?
I have an electric heater and close to it a temperature sensor. And I could use some help with controlling the heater.
The amount of power supplied to the heater can be changed at any moment. Let $0 \...
- 2,942
0
votes
0
answers
15
views
How can I prove the contraction property on a joint system of equations?
I'm studying a simple dynamic programming problem whose solution is a system of Bellman equations, and am running into some issues trying to prove the Contraction Mapping Theorem for the system as a ...
- 1
0
votes
1
answer
23
views
What is a Generalised plant (interconnected system)
I am studying control theory and I was introduced to generalised plant. I find it very difficult to find resources to explain the idea with sufficient details.
What I understand is that the ...
- 115
0
votes
0
answers
33
views
Interpretation of the $(\dot{q},\dot{p})$ Hamiltonian system in optimal control
Pontryagin's minimum principle says that for a nonlinear system $\dot{q} = f(q,u)$, the cost functional $J(q,u) = \int_{0}^{T} L(q,u)dt$ is minimized if we find $q$ and $u$ such that
$$
\dot{q} = \...
- 206
0
votes
0
answers
30
views
Optimality conditions for optimization of an expected value
I am looking to find information on the optimality conditions on a constrained optimization of the following form (assume that all functions are either affine or suitably convex/concave where ...
- 11
1
vote
1
answer
61
views
Does introducing a control input constraint lead to non-affine control? [closed]
Lets say the control input $u$ in a non-linear system $\dot x= f(x) + Bu$ is constrained. For example, $0 \leq u \leq u_{\max}$.
Does introducing such constraint on $u$ leads to a non-affine control ...
- 11
0
votes
0
answers
19
views
An intuitive view towards Pontryagin's Maximum Principle
I was going through Pontryagin's maximum principle ,in a Numerical Optimal Control course, where it is introduced under indirect methods.
I am having a bit of difficulty in appreciating why to bring ...
1
vote
1
answer
38
views
Optimal control problem with integral constraint and mid-point state condition
I am trying to solve a control problem with control $u$ and state $x$ with the following structure:
$$\max_{u(\cdot),x(\cdot)} \int_0^1 f(x(t),u(t),t)dt,$$
subject to
$$x'(t)=g(x(t),u(t),t),$$
$$\...
- 13
0
votes
0
answers
12
views
What is the optimal policy for doing serial dilutions?
So a random thought came up to me while thinking about lab work.
In a chemistry scenario, there often is a need to do dilutions, where given a stating concentration $C_0$ at volume $V_0$, and a target ...
0
votes
0
answers
42
views
Catalog of optimal control problems with analytical solutions
Is there any exhaustive reference with a list of optimal control problems with analytical solutions? perhaps starting from the easy ones, like the Brachistochrone and the Zermelo problem, and moving ...
- 233
0
votes
1
answer
99
views
Martingale property of optimal control
I am trying to solve Exercise 25.4 of Tomas Björk's Arbitrage Theory in Continuous Time.
The exercise goes as follows:
Consider the problem of minimizing
$$
\mathbb{E}\left[\int_0^T F(t, X_t^u, u_t)dt ...
0
votes
0
answers
23
views
Optimal control, measurable functions and fundamental matrix
I am studying the article Maximum Principle, Dynamic Programming, and Their Connection in Deterministic Control by X. Y. Zhou.
Given $s\in[0,1]$ and $y\in\mathbb{R}^d$ let's consider the following ...
- 185
2
votes
1
answer
131
views
What is a "high dimensional PDE" as opposed to a PDE in 1, 2, 3 dimensions?
I am reading an arxiv paper that references high dimensional PDEs. I have a picture from the text below. The claim is about " In filtering and
optimal control, we are easily interested in PDEs ...
- 1,883
0
votes
0
answers
30
views
Deriving the HJB equation in stochastic control problem
I'm reading the paper by J.Gatheral and A.Schied - "Optimal Trade Execution under
Geometric Brownian Motion in the Almgren and Chriss Framework". On page 6, the authors provide a value ...
0
votes
0
answers
21
views
$\min_{u∈U} J(u) = \int_{0}^{1} 2 (1 − u(t))x(t)dt$ s.t $x'(t) = (2u(t) − 1)x(t)$ $x(0) = 1$ $x(1) = 2$ $u(t) ∈ U := [0, 1] ∀t ∈ [0, 1]$.
Hey I have this problem where I have some questions.
We consider the following optimization problem
$\min_{u∈U} J(u) = \int_{0}^{1} 2 (1 − u(t))x(t)dt$
s.t
$x'(t) = (2u(t) − 1)x(t)$
$x(0) = 1$
$x(1) = ...
- 432
0
votes
0
answers
17
views
Questions about an optimization problem with terminal costs
So I have this problem athat we solved in class and I have some small questions.
We consider the following optimal control problem
$\min J(u) = 1/2 \int_{0}^{1} u(t)^2 dt+ 1/2 x(1)^2$
s.t.
$x(0) = x_0$...
- 432
1
vote
1
answer
48
views
Pontryagin's Maximum Principle expaination
I am having problems understanding the Pontryagin's Maximum principle.
I really dont understand the necessary conditions for minimization problem. On every website that I checked I have the impression ...
- 432
0
votes
0
answers
13
views
Optimal Control With Absolutely Continuous State Variable (not necessarily differentiable).
In the "textbook" theory of optimal control, the state variable $x(\cdot)$ is often assumed to be differentiable, or piece-wise differentiable. I am interested in a control problem in which $...
- 11
3
votes
1
answer
101
views
How can I solve following cooperative differential game?
Consider a game-theoretic model of pollution control. There are 2 players join in the game, N = {1, 2}. Each player has an industrial production site. It is assumed
that the production is proportional ...
- 74
1
vote
0
answers
45
views
Hamilton-Jacobi-Bellman equation for Levy processes
I am trying to understand a optimal investment/stochastic control Problem and derive the HJB equation for following Wealth Process
$dX^{\phi}(t)=\int_{0}^{t} X^{\phi}(s-)(r+\phi(s)(\mu-r))ds+\int_{0}^{...
- 51
0
votes
1
answer
95
views
How to verify if the kalman gain matrix K is working properly?
If I have a state space model.
$$x(k + 1) = Ax(k) + Bu(k)$$
$$y(k) = Cx(k) + Du(k)$$
And a kalman gain matrix $K$. Then, how do I know if the kalman gain matrix $K$ is properly designed for my state ...
- 2,732
1
vote
1
answer
50
views
Optimal control and Value function
Let's consider this optimal control problem:
Minimize $-x(1)$,
subject to $dx(t)/dt=x(t)u(t)$ for almost every $t \in [0,1]$,
$x(0)=0$
among all the admissible controls $u:[0,1] \to [0,1]$ such that $...
- 185
0
votes
1
answer
39
views
Adding a linear term to the quadratic cost function in the LQR controller design
I am basically trying to see the effect of adding a linear term to the conventional quadratic cost function used in the design of LQR controller for a finite horizon, free terminal state, discrete ...
1
vote
1
answer
78
views
How can I derive Bellman's principle of optimality in discrete-time?
In the context of discrete-time optimal control theory, Bellman's principle of optimality is useful for efficiently determining the control signal $\{u_k\}_{k=0}^{N-1}$ that minimizes the following ...
- 988
3
votes
1
answer
156
views
Role of verification theorems in stochastic optimal control?
I am currently working on the optimal control of certain classes of stochastic processes and I have difficulties understanding the roles of verification theorems.
My problem is the following: I am not ...
- 5,447
0
votes
0
answers
31
views
Minimization on all continuous and piecewise $C^1$ functions on $[−1, 1]$
Hi I have problems with this exercise. I really don't know how to start and how to proceed. Can someone help me?
We consider the following optimization problem in $V = \left \{ f ∈ C^1([−1, 1]\...
- 693
3
votes
1
answer
110
views
Understanding HJB equation for the infinite horizon consumption control problem
Context
Given the following maximization problem as well as wealth dynamics
$$\max\mathbb{E}\left[\int_0^{\infty} \frac{1}{\gamma} e^{-\beta t} c_t^\gamma \mathrm{d} t\right]$$
$$\mathrm{d} X_t=X_t\...
- 312
0
votes
0
answers
25
views
Can you help me on the MPC theory Teminal ingredients?
Can you help me on the MPC theory Teminal ingredients ? more in particulat on the Ellipsoid constrained
You can see the picture.
1
vote
0
answers
43
views
Rewriting an optimal control law in feed back form
We have a dynamical system:
\begin{align*}
\dot{x}&=Ax+Bu\\
x(0)&=x_{0}
\end{align*}
that is controllable. Given the cost function:
$$
J(u,t_{1})=\int_{0}^{t_{1}}u^{T}udt,\quad x(t_1)=0,
$$
it ...
- 11
3
votes
1
answer
44
views
Find optimal policy to maximize given stochastic process.
Let $x$ be a real-valued random variable with finite and constant expected value $\mathbb{E}[x] = \mu$ and $Var[x_t]=\sigma^2$. We observe outcomes of this random variable in discrete time $(x_t)_{t=1}...
- 1,382
0
votes
0
answers
27
views
Find a field of extremals
I have this exercise that we have done in class where I have problem understanding the solution.
We consider the following optimization problem $\int^{2}_{1} y'(x) + x^2 y'(x)^2 dx$. Calculate a ...
- 693
0
votes
1
answer
50
views
Questions about the Picard–Lindelöf theorem for the ODE $y(0) = y_0 (1)$ $y'(x) = αy(x)$
Hi i have questions about an exercise that we have done in class:
We consider the following ODE for a given $y_0 ∈ \mathbb{R}$ and $α ∈ \mathbb{R}$
$$
\left.\begin{gathered}
y(0) = y_0
\\
y'(x) = αy(...
- 693
0
votes
0
answers
22
views
How to convert a continuous time optimal control problem into to discrete time optimal control problem?
I have an optimal control problem and want to embed it into multi stage optimization problem by first discretizing it into discrete time optimal control problem...
Can I do it by just converting ...
1
vote
1
answer
53
views
How to derive an optimal, continuous-time linear quadratic estimator from a Luenberger state observer?
How does one derive an optimal, continuous-time linear quadratic estimator from a Luenberger state observer? I am aware of a Kalman filter, but I would like to see a derivation of an observer without ...
- 2,422
0
votes
0
answers
37
views
What does "impulse response matrices" mean here?
In this paper about differential games (DG), the authors discuss a certain class of DG:
Determine a saddle point $u(t;x_0,t_0),v(t;x_0,t_0)$ for $$J=\frac{a^2}{2}\|x_p(T)-x_e(T)\|^2_{A^TA}+\frac{1}{2}\...
- 49
2
votes
0
answers
82
views
Stability of discrete time systems.
Let a discrete time system be
\begin{align}
x[k+1]=Ax[k]
\end{align}
If the system in Eq.1 is stable then always it will satisfy the Lyapunov equation as described below.
Let the Lyapunov function be $...
1
vote
1
answer
43
views
Question on nonlinear optimal control problem
Problem: Given a discrete equation of state
$$x^{k+1} = x^k -u^k,\ u^k \ge 0.$$
Our goal is to drive in $N$ steps the system to the origin, from $x^0$ and minimize the cost function
$$J(x,u) := \sum_{...
- 707
0
votes
0
answers
23
views
epsilon balls and 0- and 1- norms in optimal control
Please consider the following excerpt from Calculus of Variations and Optimal Control Theory, A Concise Introduction by Daniel Liberzon
Here the space $\mathcal{C}^k$ is the space of k times ...
- 974
0
votes
0
answers
26
views
How to solve the optimal control problem?
Optimal control problem:
$\dot{x}=rx-\alpha u $
$J=\int_{0}^{\infty}e^{-\rho t}((b-\frac{u}{2})u-cx)dt$, $\mapsto Max$
Where $r,\alpha, b,c, \rho \in R$ and all parameters are not negative.
where as $...
- 74
0
votes
0
answers
16
views
Observability of plates - Proof of the improvement (condition Observability)
hey guys I was learning about the Observability of plates and I came across integral equality I didn't get how they got into it from the integral
here is the ...
0
votes
0
answers
16
views
why does maximized utility of consumption(merton problem) exist?
Agent controls his proportion of wealth invested in the stock $ \alpha_t $ and his consumption rate $c_t$.
Dynamic of wealth:$ dX_t=X_t[(\alpha_t(u-r)+r)dt+\alpha_t \sigma_t dW_t]-c_t dt $
value ...
0
votes
0
answers
47
views
Is $L^{\infty}([0, 1]; \mathbb{R}^{n})$ densely embedded in some Hilbert space? [duplicate]
I'm solving a control problem and I have a question. Is it possible to show that $L^{\infty}([0, 1]; \mathbb{R}^{n})$ is densely embedded in some Hilbert space?
0
votes
0
answers
35
views
The conditions of optimization task
Could you please help me to solve the task using optimization method?
The factory produces three types of glue. And four types of chemicals are used for its production: starch, gelatin, alum and chalk....
- 63
0
votes
0
answers
15
views
Rank condtion and stationary point for constrained systems
I am trying to understand how is Hamiltonian function defined. For the optimization problem, $\min L(x,u) s.t. f(x,u)=0$, a necessary condition for a minimum is that $\left[\begin{array}{c}d L \\ d f\...
- 87
-1
votes
2
answers
27
views
Solving a diffrential equation with deriviative of more than one dependent variable
How should I go about solving a differential equation of the form:
$ \frac {d}{dR}(f_1(R)g_1(R)+f_2(R) g_2 (R))=0$
where $f_1(R)$ and $f_2(R)$ are known.
I am trying to solve for $g_1(R)$ and $g_2(R)$...
1
vote
1
answer
122
views
Solution to an algebraic Riccati equation with complex matrices
I am trying to find the analytical solution for the following Riccati equation:
$$
0 = F + W^\dagger P(t) + P(t) W + P(t)X P(t).
$$
In my particular problem I know that it has a solution. In this ...
- 125
1
vote
0
answers
56
views
Solve discrete Algebratic Riccati Equation if S is non-square - How?
I have the state-space model
$$\dot x = Ax + Bu + W$$
$$y = Cx + Du + E$$
where $E\in \mathbb{R}^{n \times (N-1)}$ is a noise vector and $W\in \mathbb{R}^{p \times (N-1)}$ a disturbance vector.
What I ...
- 2,732
2
votes
0
answers
33
views
Viscosity supersolution of HJB equation
I'm having some troubles solving a question in an exercise. The set-up is the following:
Let $\sigma:\mathbb R^d \rightarrow \mathbb R^{d\times d}$ be a $\mathcal C^2$ map which is bounded and has ...
- 21
0
votes
1
answer
42
views
What is the ratio of a transfer function $G(s)$?
If I have a transfer function $G(s) = \frac{Y(s)}{U(s)}$ where the $G(s)$ is the ratio between the amplitude of $Y(s)$ and $U(s)$. In what unit is the ratio?
Let's say that we have a input signal $u(t)...
- 2,732
1
vote
0
answers
30
views
Dynamic Nash Bargaining Solution
Static Game
Let $i \in \{1, 2\}$ denote a player.
Each player can execute an action $a_i \in A_i$, where $A_i \subseteq \mathbb R$ denotes the set of feasible actions.
Given a pair of actions $a = (...
- 737