Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [optimal-control]

Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. (Def: http://en.m.wikipedia.org/wiki/Optimal_control)

0
votes
0answers
15 views

Bellman's Principle of Optimality

I'm currently reading Pham's Continuous-time Stochastic Control and Optimization with Financial Applications however I'm slightly confused with the way the Dynamic Programming Principle is presented. ...
0
votes
1answer
21 views

How to find a transformation matrix which will make the system a chain of integrators?

Consider a system of the form $$\dot{x}(t)=Ax(t)+Bu(t)+\phi(t)+D(t)$$ I have $$\dot{x}(t)=\begin{bmatrix} -p_1 &G_b & 0 & 0 &0 \\ 0& -p_2 & p_3 & 0 & 0\\ 0& ...
-1
votes
0answers
10 views

How to control heat equation using HJB equation? [on hold]

I have the heat equation u_xx = u_t ,u(0,t)=u(1,t)=0 ,u(x,0)=sin⁡(2πx) so i will use HJB to control this but I don't how to obtain the state space and cost function to use HJB ?
0
votes
0answers
17 views

How to solve Heat equation by Riccati equation ?

I turned the heat equation into a product of two linear differential equations and I think of using Adomian decomposition method for solving it ? it is possible ?
1
vote
1answer
21 views

What is the cost function (performance index) for 2nd order ODE systems?

I have the following 2nd order linear system with appropriate initial conditions. $$\textbf{X}''(t)+\textbf{A}(t)\textbf{X}'(t)+\textbf{B}(t)\textbf{X}(t)=\textbf{F}(t)+\textbf{C}(t)\textbf{U}(t)$$ $\...
1
vote
1answer
52 views

How would I compute this matrix in Matlab?

I'm trying to compute the H$_\infty$ norm of a matrix using the paper: Bruinsma, N. A.; Steinbuch, M., A fast algorithm to compute the $H{\infty}$-norm of a transfer function matrix, Syst. Control ...
1
vote
0answers
17 views

Guess and verify: verification theorem for Hamilton-Jacobi-Bellman equation

Let $t \in [0,\infty)$ denote time, $x(t) \in X \subset \mathbb R_+$ the state and $u(t) \in \mathbb R_+$ the control. Consider the following optimal control Problem \begin{align} &V(x_0) = \max_{...
0
votes
0answers
20 views

How to design a Robust observer for a 2D system

Consider the second order system given by $\dot{x}=Ax+Bw(t)$, where $x\in\mathbb{R}^2$, $$A = \begin{bmatrix} {0},{6}\\ {-1} {-6} \end{bmatrix}, \quad B = \begin{bmatrix} {0}\\{1}\end{bmatrix}...
0
votes
0answers
33 views

Dynamic programming's principle of optimality as an abstract construct

In dynamic programming, the principle of optimality (refer to Bertsekas's Optimal Control, volume 1, page 18) is a statement that says: For any optimal policy $\pi$ , we always have a suboptimal ...
1
vote
0answers
20 views

Price of a stochastic game between an agent and the market

In the article Pricing via utility maximization and entropy from Richard Rouge and Nicole El Karoui, they define the value function of the optimization problem as \begin{align} V(x,C) = \dfrac{1}{\...
0
votes
0answers
9 views

linearizing dynamics about non fixed point for LQR implementation.

I am trying to implement LQR control for the cart pole system. I am curious if I can maintain a constant non-zero pole angle. So, I need to linearize my dynamics about my goal state. I know we can use ...
0
votes
0answers
18 views

difference between HJ and HJB equation

This question might be very simple: what is the difference between Hamilton-Jacobi (HJ) equation and Hamilton-Jacobi-Bellman (HJB) equation, especially in optimal control theory? My personal ...
1
vote
0answers
19 views

Equivalent formulations of stochastic HJB equation

I have some trouble understanding stochastic HJB equations. There are basically two forms of this equation that I have encountered in books, lecture notes etc... (one-dimensional case) 1) $rv(x)=\pi (...
0
votes
1answer
40 views

How can I add find the gain from root locus and poles?

I try to find the P-gain from a root locus plot where I know the poles. Assume that we got a reference model: $$G(s) = \frac{\omega_n^2}{s^2 + 2\zeta \omega_n s + \omega_n^2 }$$ Where $\zeta$ and $\...
0
votes
1answer
30 views

Injectivity trajectory to set singular control to zero

Consider a control system of the form $$ \dot x(t) = X(x(t)) + u(t)\, Y(x(t)) \qquad \qquad (*) $$ where $X,Y$ are two smooth vector fields, and $u$ is the (bounded measurable) control function. ...
3
votes
2answers
48 views

Optimal stopping time for coin toss with unkown bias

I am working on a question that involves uncertainty and decision making, but I realized I am not making progress for a long time. That is why I formulated a more basic problem in the hope that I can ...
1
vote
0answers
32 views

Reference: control theory

I'm looking for suggestions on books or reviews on control theory, if possible written in a fairly modern language and not excessively technical on the math side. In particular, I would need the ...
1
vote
1answer
17 views

Stability proof of nominal MPC with terminal cost and constraint

While going trough these slides, I wasn't able to make sense of the following on slide 32: (if only providing the url to the slides is not ok I will edit the question, but doing it like that saves a ...
0
votes
1answer
33 views

Finding the optimal solution of a specific parametric performance index with constraints

Consider the following performance index: $$J=\cfrac{1}{2}u_1^2+\cfrac{1}{2}u_2^2+\cfrac{1}{2}u_3^2+p_1\cdot u_1+p_2\cdot u_2+p_3\cdot u_3$$ Suppose $u_1$, $u_2$ and $u_3$ are the design variables and ...
0
votes
0answers
18 views

$H_\infty$ norm convergence (control systems)

So I have a question about proving the convergence (or rather a bound) of an optimization problem. I want to minimize the following function: $$\gamma := \left\|\frac{Q[A-B(x)C]}{\Re \{A \}} \right\|...
0
votes
1answer
42 views

Pontryagin Principle

Is there any reference from which I can clearly understand The Pontryagin Maximum Principle (with some clear examples). I need it to apply to dynamical systems (differential equations) and P.D.E (...
8
votes
1answer
84 views

Role of the weight matrix $M$ in $x^T M u$ in the LQR cost function

I wonder what the role of the weight matrix $M$ is in the performance index $$J = \int_0^{t_f}{\left( x^T Q x + u^T R u + x^T M u \right) \mathrm d t}$$ for an optimal control problem where $$\dot ...
2
votes
1answer
61 views

solving an optimal stopping problem

I am currently going through problems in Oksendal's intro SDE and stuck with this problem. I was wondering if I could get some help with it. I would sincerely appreciate if you would express out all ...
0
votes
1answer
29 views

Can I build an adaptive controller by using an ODE solver and a 3D graphics engine? [closed]

Let's assume that you're using a 3D graphics engine with built in physics. You create a inverted pendelum in a 3D designing software, e.g Blender, and then import the model into your 3D grapics engine ...
1
vote
1answer
32 views

Calculus of variations with inverse of derivative composited within?

Suppose $s \in S \subset \mathbb{R}^+$ with c.d.f. $F(s)$. A real-valued function $\beta(s) \leq s$ is determined by a first order condition $\gamma'(s - \beta(s)) = g(F(s)) \in \mathbb{R}$, where $\...
2
votes
0answers
37 views

What's wrong with this robust control scheme?

I'm learning how to control a double integrator with $H_\infty$. my model is simply $ \dot{r} = v $ $ \dot{v} = F/m $ $ r(t_0) = 0$ m, $v(t_0) = 0 $ m/s, $m = 1000 $ kg so I want to be able to ...
0
votes
0answers
24 views

Control invariant set for not pointwise-in-time constraints

Given is a system of the form $$ x_{k+1} = Ax_k+Bu_k, \ k\geq 0 $$ with $x_0$ known, $x_k \in \mathbb{R}^n,u_k \in \mathbb{R}^m$ and $A,B$ of appropriate dimensions. Let $u = \begin{bmatrix}u_0 \\ \...
1
vote
1answer
30 views

Saturation limit compared to constrained limit

I have a simple question. What's the difference in behaviour between saturation limit and constrained limit in control theory? We say that we got this objective function: $$J_{min} = \frac{1}{2}x^...
0
votes
1answer
27 views

How can I handle the delays in Generalized/Model Predictive Control?

I trying to handle delays in a model who is poorly damped but I haveing som issues to estimate its parameters due to the delay. Assume that we got a state space model, which is poorly damped: $$x(k+...
2
votes
0answers
24 views

What's the idea behind having internal integration in predictive control?

According to lots of books about predictive control, they recommend to having internal integration inside the model. For example if we have a state space model: $$x(k+1) = Ax(k) + Bu(k) \\ y(k) = Cx(...
0
votes
0answers
19 views

Create a Kalman filter from ARMAX model?

Assume that I have a ARMAX model: $$A(z)y(t) = B(z)u(t) + C(z)e(t)$$ I going to use the Algebraic Riccati Equation(ARE) to find the LQR control law $L$ by selecting the weighting matrices $Q$ and $R$...
0
votes
2answers
95 views

What's the difference between Generalized Predictive Control and Model Predictive Control?

As I know, the Generalized Predictive Control(GPC) is older than Model Predictive Control(MPC). But what is the real difference between them? I know that GPC contains some kind of system ...
0
votes
1answer
20 views

Formalizing a Parameter Selection Problem in Machine Learning

I have a simple problem, I have an array (say of length 300), which gives me the fraction of importance of some value. I plot it and see that the value becomes flat after some index. So, I select ...
1
vote
0answers
76 views

tracking controller with fixed control bandwidth in h infinity framework

I want to synthesize an $H_\infty$ tracking controller for a second-order system. The system is subject to noise on position, velocity, and acceleration. If we look at the typical framework with ...
0
votes
0answers
16 views

How do I choose the polynomials for a stochastic filter? - Transfer functions + Extended Least Square

I'm buildning a Mimimum Variance Controller(MVC) but I having som trouble to select the stochastic filter. First of all! To build a MVC, you need a ARMAX model, in other words polynomial who look ...
2
votes
0answers
28 views

What's the applications of Minimum Variance Controller?

I going to show how to create a Minimum Variance Controller(MVC) and then ask what's the applications of MVC. First! Let's say that we have a stochastic transfer function model, ARMAX in other words. ...
1
vote
0answers
35 views

How can I choose the disturbance model if I know the plant and controller - Transfer functions

I going to select the disturbance models $C_f$ and $H$. I know my plant $P$ and the controller $C_b$. I also know that the disturbance $d$ is step formed. The noise $v$ is $v = 0$. Question: How ...
0
votes
0answers
22 views

Integral action or correction factor on disturbance?

I wonder what's the difference between having a integral action or correction factor when it comes to disturbances? Ofc I know how to apply then, the reason for this question is: What's suits best ...
1
vote
2answers
30 views

How do I find a controller for non-minimum phase systems?

Assume that we have a transer function $G(s) = \frac{B}{A}$ which has stable poles, but unstable zeros. We use the controller $Q(s) = \frac{A}{B} = G^{-1}(s)$ and we want that the loop transfer ...
1
vote
0answers
35 views

How do I further constrain the Hamilton-Jacobi-Bellman equation to find the optimal control?

I am attempting to find an optimal feedback control using the HJB equation. I am trying to stabilize an n-dimensional state vector $x$ using a scalar control $u$ as follows: $$\dot{x} = Ax+uBx$$ ...
1
vote
1answer
62 views

How do I place the poles and zeros form a disired system? Adaptive control

If I have a transfer function of a system $G(s)$ $$G(s) = \frac{4 - 2s}{4 + 0.8s + s^2}$$ $G(s)$ has the poles and zeros and is a stable system. And the step answer look like. It has a delay as you ...
2
votes
0answers
77 views

Numerical methods for minimum time optimal problems

I have the next optimal problem and I want to make your numerical approximation. Reviewing some references, it is mentioned that numerical methods must also be used to solve the dynamic equation and ...
1
vote
1answer
244 views

What's the difference between non-minimum phase systems and minimum phase systems?

I wonder if you can explain what's the difference between non-minimum phase systems and minimum phase systems? How can I recognize them in bode/time plots? Is this a minimum phase system? $$G(s) = \...
0
votes
0answers
61 views

Optimal control problem (constant magnitude acceleration)

A particle in $\mathbb R^2$ begins at initial position $(x_0, y_0)$ and velocity $(u_0, v_0)$. It must eventually reach a target position $(x_1, y_1)$ and velocity $(u_1, v_1)$. The acceleration of ...
1
vote
1answer
81 views

Time-optimal control of a car

I'm having trouble with the following problem: $$\begin{array}{ll} \text{minimize} & J := \displaystyle\int_{t_0}^{t_f} 1 \,\mathrm{d}t\\ \text{subject to} & \dot{v}(t)=-9.8\sin(\theta)+u(t)-\...
1
vote
3answers
54 views

What's the point of creating discrete control laws for analog processes?

Assume that we have a state space model of a real system e.g mass-spring system. $$\dot x = Ax + Bu$$ Then we want to implement this in a micro controller. The controller has a sampling rate of $h = ...
1
vote
1answer
58 views

What is the minimum time from point A to point B?

am working a bit on the theory of optimal control, and I have had a couple of doubts about how I should choose the control variable to minimize travel time. Consider the control problem to reduce the ...
2
votes
0answers
28 views

Redesigning MPC augmented state for integrating action

In model predictive control for a system $$ \begin{cases} x_m(k+1)=A_m x_m(k)+B_m u(k)+\xi_k\\ y_m (k)=C_m x_m(k)+\eta(k) \end{cases} $$ where $u$ is the manipulated variable (dimension $y$ is the ...
5
votes
0answers
74 views

Maximizing expected reward at final time via dynamic programming

Consider a state $x_t$ that probabilistically evolves over time according to a controlled Markov chain, i.e., according to known probabilities $$\mathbb P(x_{t+1}=x' \,|\, x_t=x,a_t=a)$$ where $...
0
votes
0answers
15 views

How to obtain a relation between the cost and the payoff in optimal control problems?

I have an optimal control problem to maximize the function $J =\int_0^1 x(t)-\alpha u^2(t) dt$ subjects to the system $dx(t)/dt = f(x(t),u(t),t)$ and the initial/final states. The system $dx(t)/dt$ ...