Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [optimal-control]

Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. (Def: http://en.m.wikipedia.org/wiki/Optimal_control)

0
votes
0answers
30 views

Solution for first order ODE with discontinuous right hand side

I'm now approaching for the first time to first order differential equations with discontinuous right hand side. Let $A\subset \mathbb{R}\times\mathbb{R}^n$ and $f=f(t, x):A\to \mathbb{R}^n$. Fix $(...
0
votes
0answers
30 views

How to minimize the next functional using the Pontryagin Maximum Principle?

Best regards, I am asked to minimize the next functional $T(v)=\int_{A}^{B}\frac{dx}{v(x)}$, with $v\neq 0\text{ and } v\in V$, where $V=\{v\in C([A,B]):v(A)=v(B)=0\}$. If we assume that there ...
0
votes
1answer
29 views

A simple quadratic optimizer for only constraints on input

I'm going to implement an quadratic optimizer with C for embedded systems. I will do that because I need speed. But I have some trouble to find a quadratic optimizer for C that works with embedded ...
0
votes
0answers
29 views

Software recommendation needed for optimal control

My subject is optimal control of PDE and I want to put optimal control problems in Matlab to be solved but I don't know how to do this. I would be grateful if you could give me some references in that ...
0
votes
2answers
36 views

Is there any rule of thumb when it comes to selecting control/predict horizon for MPC?

I have a simple question: Is there any rule of thumb when it comes to selecting control/predict horizon for MPC? Normaly I set control and predict horizon equals, but I have heard that's not good ...
-1
votes
0answers
17 views

How to add inequality condition in solving differential system with maple?

My work is following pontryagin maximum principle. But i have a problem solving the differential equation system, where there are 6 differential equations with 6 initial conditions. Everything works ...
0
votes
0answers
13 views

Value iteration method of optimal stopping

In Lawler's introdcution to Stochastic process p89~93, the value iteration method is given for homogeneous Markov chain: $u_1(x)$ equal to the payoff function $f(x)$ if $x$ is an absorbing state and ...
1
vote
0answers
37 views

Optimal control problem with a path constraint which involves controls at two distinct time points

I am faced with an optimal control problem in continuous time which includes a path constraint which involves controls at two distinct points in time. I do not know how to approach this problem. I do ...
1
vote
1answer
23 views

Unit of time and normalization of time preference rates

Consider an infinite horizon cake eating differential game described by \begin{align} &\max_{u_1(t)} \int_0^\infty{e^{-r_1 t}\ln(u_1(t))dt}\\ &\max_{u_2(t)} \int_0^\infty{e^{-r_2 t}\ln(u_2(t))...
1
vote
1answer
34 views

Numerical solution of non-linear first order partial differential equation (HJB)

I am trying to solve a simple optimal control problem using the Hamilton-Jacobi-Bellman equation, numerically in Python. This is proving to be rather difficult as I end up having to solve the ...
0
votes
0answers
54 views

Formulation for calculus of variation with state-space constraint

I'm stuck on this question, let $B = \{x\in \mathbb{R}^n:|x|\leq 1\}$ be the unit ball in $\mathbb{R}^n$, consider the following minimizing problem $$ \inf_{x(\cdot) \in \mathcal{A}} \int_0^\infty e^{-...
0
votes
0answers
20 views

Is dynamic programming suitable for a specific optimization problem?

Let $c,\,\mathcal{P}_0,\,\mathcal{P}_1,\,\mathcal{P}_2,\ldots$ be a sequence of positive real numbers. Let $N\in\{1,\,2,\,3,\ldots\}$ and let $t\in\{0,\,1,\,2,\ldots\}$, with $N$ and $t$ fixed. ...
0
votes
1answer
33 views

Questions about LQG with full information

I have implemented LQG in MATLAB software. But, now I do not know how to determine the value of optimal cost. Each way of calculating cost, returns a different value. Which one should I trust to ...
0
votes
0answers
25 views

The Riccati equation and its asymptotic behavior

Consider matrices $A\in\mathbb{R}^{n\times n},B\in\mathbb{R}^{n\times m}$, a positive semidefinite symmetric matrix $Q\in\mathbb{R}^{n\times n}$ and a positive definite symmetric matrix $R\in\mathbb{R}...
1
vote
1answer
43 views

Optimal control problem with fixed endpoint; what I am doing wrong?

I am trying to solve the following problem (I choose a numerical example). The state $x$ is governed by a differential equation linear in $x\in [0,1]$. Two variables, $\theta \in[0,1]$ and $\rho\in[0,...
0
votes
0answers
26 views

Time-Discretize a Linear Quadratic OCP (Bolza Function)

I'm trying to formulate an optimal control problem based off of this given Minimum-Time Cost Function: $$J(t_f)=\frac{1}{2}[x(t_f)-x_{des}(t_f)]^TP_f[x(t_f)-x_{des}(t_f)] + \frac{1}{2}\int_{t_0}^{t_f}(...
1
vote
0answers
68 views

Bellman's Principle of Optimality

I'm currently reading Pham's Continuous-time Stochastic Control and Optimization with Financial Applications however I'm slightly confused with the way the Dynamic Programming Principle is presented. ...
0
votes
1answer
27 views

How to find a transformation matrix which will make the system a chain of integrators?

Consider a system of the form $$\dot{x}(t)=Ax(t)+Bu(t)+\phi(t)+D(t)$$ I have $$\dot{x}(t)=\begin{bmatrix} -p_1 &G_b & 0 & 0 &0 \\ 0& -p_2 & p_3 & 0 & 0\\ 0& ...
0
votes
0answers
24 views

How to solve Heat equation by Riccati equation ?

I turned the heat equation into a product of two linear differential equations and I think of using Adomian decomposition method for solving it ? it is possible ?
1
vote
1answer
37 views

What is the cost function (performance index) for 2nd order ODE systems?

I have the following 2nd order linear system with appropriate initial conditions. $$\textbf{X}''(t)+\textbf{A}(t)\textbf{X}'(t)+\textbf{B}(t)\textbf{X}(t)=\textbf{F}(t)+\textbf{C}(t)\textbf{U}(t)$$ $\...
1
vote
1answer
52 views

How would I compute this matrix in Matlab?

I'm trying to compute the H$_\infty$ norm of a matrix using the paper: Bruinsma, N. A.; Steinbuch, M., A fast algorithm to compute the $H{\infty}$-norm of a transfer function matrix, Syst. Control ...
1
vote
0answers
23 views

Guess and verify: verification theorem for Hamilton-Jacobi-Bellman equation

Let $t \in [0,\infty)$ denote time, $x(t) \in X \subset \mathbb R_+$ the state and $u(t) \in \mathbb R_+$ the control. Consider the following optimal control Problem \begin{align} &V(x_0) = \max_{...
0
votes
0answers
23 views

How to design a Robust observer for a 2D system

Consider the second order system given by $\dot{x}=Ax+Bw(t)$, where $x\in\mathbb{R}^2$, $$A = \begin{bmatrix} {0},{6}\\ {-1} {-6} \end{bmatrix}, \quad B = \begin{bmatrix} {0}\\{1}\end{bmatrix}...
0
votes
0answers
49 views

Dynamic programming's principle of optimality as an abstract construct

In dynamic programming, the principle of optimality (refer to Bertsekas's Optimal Control, volume 1, page 18) is a statement that says: For any optimal policy $\pi$ , we always have a suboptimal ...
1
vote
0answers
23 views

Price of a stochastic game between an agent and the market

In the article Pricing via utility maximization and entropy from Richard Rouge and Nicole El Karoui, they define the value function of the optimization problem as \begin{align} V(x,C) = \dfrac{1}{\...
0
votes
0answers
17 views

linearizing dynamics about non fixed point for LQR implementation.

I am trying to implement LQR control for the cart pole system. I am curious if I can maintain a constant non-zero pole angle. So, I need to linearize my dynamics about my goal state. I know we can use ...
0
votes
0answers
27 views

difference between HJ and HJB equation

This question might be very simple: what is the difference between Hamilton-Jacobi (HJ) equation and Hamilton-Jacobi-Bellman (HJB) equation, especially in optimal control theory? My personal ...
1
vote
0answers
24 views

Equivalent formulations of stochastic HJB equation

I have some trouble understanding stochastic HJB equations. There are basically two forms of this equation that I have encountered in books, lecture notes etc... (one-dimensional case) 1) $rv(x)=\pi (...
0
votes
1answer
46 views

How can I add find the gain from root locus and poles?

I try to find the P-gain from a root locus plot where I know the poles. Assume that we got a reference model: $$G(s) = \frac{\omega_n^2}{s^2 + 2\zeta \omega_n s + \omega_n^2 }$$ Where $\zeta$ and $\...
0
votes
1answer
32 views

Injectivity trajectory to set singular control to zero

Consider a control system of the form $$ \dot x(t) = X(x(t)) + u(t)\, Y(x(t)) \qquad \qquad (*) $$ where $X,Y$ are two smooth vector fields, and $u$ is the (bounded measurable) control function. ...
3
votes
2answers
54 views

Optimal stopping time for coin toss with unkown bias

I am working on a question that involves uncertainty and decision making, but I realized I am not making progress for a long time. That is why I formulated a more basic problem in the hope that I can ...
1
vote
1answer
54 views

Reference: control theory

I'm looking for suggestions on books or reviews on control theory, if possible written in a fairly modern language and not excessively technical on the math side. In particular, I would need the ...
1
vote
1answer
19 views

Stability proof of nominal MPC with terminal cost and constraint

While going trough these slides, I wasn't able to make sense of the following on slide 32: (if only providing the url to the slides is not ok I will edit the question, but doing it like that saves a ...
0
votes
1answer
38 views

Finding the optimal solution of a specific parametric performance index with constraints

Consider the following performance index: $$J=\cfrac{1}{2}u_1^2+\cfrac{1}{2}u_2^2+\cfrac{1}{2}u_3^2+p_1\cdot u_1+p_2\cdot u_2+p_3\cdot u_3$$ Suppose $u_1$, $u_2$ and $u_3$ are the design variables and ...
0
votes
0answers
19 views

$H_\infty$ norm convergence (control systems)

So I have a question about proving the convergence (or rather a bound) of an optimization problem. I want to minimize the following function: $$\gamma := \left\|\frac{Q[A-B(x)C]}{\Re \{A \}} \right\|...
0
votes
1answer
48 views

Pontryagin Principle

Is there any reference from which I can clearly understand The Pontryagin Maximum Principle (with some clear examples). I need it to apply to dynamical systems (differential equations) and P.D.E (...
8
votes
1answer
91 views

Role of the weight matrix $M$ in $x^T M u$ in the LQR cost function

I wonder what the role of the weight matrix $M$ is in the performance index $$J = \int_0^{t_f}{\left( x^T Q x + u^T R u + x^T M u \right) \mathrm d t}$$ for an optimal control problem where $$\dot ...
3
votes
1answer
67 views

solving an optimal stopping problem

I am currently going through problems in Oksendal's intro SDE and stuck with this problem. I was wondering if I could get some help with it. I would sincerely appreciate if you would express out all ...
0
votes
1answer
35 views

Can I build an adaptive controller by using an ODE solver and a 3D graphics engine? [closed]

Let's assume that you're using a 3D graphics engine with built in physics. You create a inverted pendelum in a 3D designing software, e.g Blender, and then import the model into your 3D grapics engine ...
1
vote
1answer
32 views

Calculus of variations with inverse of derivative composited within?

Suppose $s \in S \subset \mathbb{R}^+$ with c.d.f. $F(s)$. A real-valued function $\beta(s) \leq s$ is determined by a first order condition $\gamma'(s - \beta(s)) = g(F(s)) \in \mathbb{R}$, where $\...
2
votes
0answers
37 views

What's wrong with this robust control scheme?

I'm learning how to control a double integrator with $H_\infty$. my model is simply $ \dot{r} = v $ $ \dot{v} = F/m $ $ r(t_0) = 0$ m, $v(t_0) = 0 $ m/s, $m = 1000 $ kg so I want to be able to ...
0
votes
0answers
29 views

Control invariant set for not pointwise-in-time constraints

Given is a system of the form $$ x_{k+1} = Ax_k+Bu_k, \ k\geq 0 $$ with $x_0$ known, $x_k \in \mathbb{R}^n,u_k \in \mathbb{R}^m$ and $A,B$ of appropriate dimensions. Let $u = \begin{bmatrix}u_0 \\ \...
1
vote
1answer
31 views

Saturation limit compared to constrained limit

I have a simple question. What's the difference in behaviour between saturation limit and constrained limit in control theory? We say that we got this objective function: $$J_{min} = \frac{1}{2}x^...
0
votes
1answer
27 views

How can I handle the delays in Generalized/Model Predictive Control?

I trying to handle delays in a model who is poorly damped but I haveing som issues to estimate its parameters due to the delay. Assume that we got a state space model, which is poorly damped: $$x(k+...
2
votes
0answers
24 views

What's the idea behind having internal integration in predictive control?

According to lots of books about predictive control, they recommend to having internal integration inside the model. For example if we have a state space model: $$x(k+1) = Ax(k) + Bu(k) \\ y(k) = Cx(...
0
votes
0answers
20 views

Create a Kalman filter from ARMAX model?

Assume that I have a ARMAX model: $$A(z)y(t) = B(z)u(t) + C(z)e(t)$$ I going to use the Algebraic Riccati Equation(ARE) to find the LQR control law $L$ by selecting the weighting matrices $Q$ and $R$...
0
votes
2answers
141 views

What's the difference between Generalized Predictive Control and Model Predictive Control?

As I know, the Generalized Predictive Control(GPC) is older than Model Predictive Control(MPC). But what is the real difference between them? I know that GPC contains some kind of system ...
0
votes
1answer
21 views

Formalizing a Parameter Selection Problem in Machine Learning

I have a simple problem, I have an array (say of length 300), which gives me the fraction of importance of some value. I plot it and see that the value becomes flat after some index. So, I select ...
1
vote
0answers
83 views

tracking controller with fixed control bandwidth in h infinity framework

I want to synthesize an $H_\infty$ tracking controller for a second-order system. The system is subject to noise on position, velocity, and acceleration. If we look at the typical framework with ...
0
votes
0answers
16 views

How do I choose the polynomials for a stochastic filter? - Transfer functions + Extended Least Square

I'm buildning a Mimimum Variance Controller(MVC) but I having som trouble to select the stochastic filter. First of all! To build a MVC, you need a ARMAX model, in other words polynomial who look ...