Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [optimal-control]

Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. (Def: http://en.m.wikipedia.org/wiki/Optimal_control)

0
votes
1answer
22 views

LQR with derivative-dependent performance?

Given a standard LTI system with $$ \dot{x} = A x + B u $$ The standard LQR finds the control gain $K$ of the state feedback $u = -Kx$ such that $$ J_1 = \int_0^\infty \big( x^T Q x + u^T R u \big) ...
0
votes
0answers
36 views

How can I design a (PID) Controller if I don't have a reference signal?

I have been trying to control lateral and longitudinal movement of a robot for an autonomous lane keeper project. I have no problem with the lateral movement, however I couldn2t figure out exactly how ...
0
votes
0answers
22 views

Model reduction of estimated state space models - System identification

Assume that we have a dynamical model in form of this simple transfer function $$G(s) = \frac{1}{2s^2 + 5s + 4}$$ G = tf(1, [2 5 4]) We do a step response with ...
0
votes
0answers
33 views

How can I reduce noise from measurement without a Kalman Filter?

I'm going to create an adaptive Model Predictive Controller (MPC). The model is a state space model. Due to noise, it's very difficult to determine the model order. I'm using subspace identification ...
1
vote
1answer
42 views

Beginner's question about fuel control of a rocket

I am very new to control and mostly just reading Bellmann's stuff. He has some nice examples and writes really clearly, although there are times when his notation gets a little crazy. Does anyone ...
0
votes
0answers
15 views

Deviation of Optimal Policy in Simulated system from Optimal Policy in real system

Say I have a real system dynamics $x_{t+1} = f(x_t,u_t)$, and I have a simulated system dynamics $x_{t+1} = f'(x_t,u_t)$, where $f'\approx f$. Suppose $\hat\phi$ is the optimal control policy under $f'...
0
votes
0answers
22 views

Proving an obvious property of a stochastic control problem

Suppose, $X$ is a diffusion that evolves as $$dX_t = C_t dB_t$$ where $B_t$ is a standard Brownian motion and $(C_t)_{t \ge 0}$ is a $(B_t)$ measurable stochastic process. Let $\mathcal C$ be the set ...
0
votes
0answers
10 views

Stochastic Control with non-continuous Scrap Value

Is there a way that the standard stochastic control problem with HJB equation applies also when the value function is given by $$ J(t,x,u) = max_{u(t)} \int_t^T f(x(s),u(s)) ds + g(x(T)) \\ s.t. dx_t= ...
4
votes
0answers
46 views

Example of a toy ODE control problem for chemistry/chemical engineering application?

I work in control theory. I have a student who is interested in chemistry and chemical engineering problems. I thought it would be somewhat easy to find such a problem, but apparently coming up with ...
0
votes
1answer
32 views
1
vote
0answers
22 views

Optimal Control: Proof of Adjoint Lagrange method?

Consider an optimal control problem: Let $x_0\in\mathbb{R}^n$, $f\in C^{0,1}([0,T]\times (\mathbb{R}^n\times \mathbb{R}^m),E)$ be bounded, let the state equation be $$\dot{x}(t) = f(t,x(t),u(t)),\, t\...
1
vote
0answers
27 views

Solution to Average of Several Trails of Dicrete Time LQR with Noise

The solution to discrete time finite horizon LQR problem is well studied. We have the linear system $$x_{k+1}=A x_{k}+B u_{k}+w_k$$ where $w_k$ is a random variable with mean $0$ and finite second ...
1
vote
0answers
35 views

Linear quadratic regulator via least squares

In this set of slides, the finite horizon LQR problem is stated as a least-squares problem (slide 11), and using a naive method (e.g., QR factorization), the cost to solve this problem is $O(N^3nm^2)$ ...
0
votes
0answers
33 views

Optimal basis generation using simplex

Given the objective function $\sum_{i=0}^{i=n} t_i$ (which I want to minimize), constraints $At = u, t \geq 0$ where $A \in m \times n$, and $ n>m$, I'm trying to determine all of the possible ...
0
votes
1answer
29 views

Model Predictive Control: Why the horizon size, $N$, must be equal or larger than 2?

If you read "Nonlinear Model Predictive Control" by L. Grune and J. Pannek (and anywhere else), everyone says that the prediction horizon size $N$ must be larger or equal to $2$,$ N\geq2$. Why?
1
vote
1answer
50 views

How can I solve the discrete algebraic Riccati equations?

I have heard that Schur decomposition $$A = USU^{-1}$$ can be used to solve discrete algebraic Riccati equations $$X = A^T X A -(A^T X B)(R + B^T X B)^{-1}(B^T X A) + Q$$ and also continuous ...
1
vote
1answer
53 views

Is LQR obsolete compared to non constrained MPC?

I have heard that LQR and MCP have common similarities. The difference is that MPC is using QP-programming and LQR using Riccati Equations. With QP-programming, constraints can be applied. If we ...
2
votes
0answers
57 views

Discrete-time linear control with linear state/input constraints

Given a controllable discrete-time linear system $x(k+1) = A x(k) + B u(k)$ the input sequence leading from state $x_0$ to $x_f$ is given by $C^{-1} (x_f - A^n x_0)$ where $C$ is the controllability ...
3
votes
1answer
29 views

Linear Quadratic Regulator Matrices Positivity

Shortly, the LQR problem says that: for $\begin{cases} x'=Ax+Bu \\ x(t_0)=x_0\end{cases}$ find: $$\min_{u\in L^2(t_0,T; \mathbb{R}^m)} J(u)=\frac{1}{2}\left\{\int_{t_0}^T x^TQx+u^TRu+2x^TNu\ dt + x(T)...
0
votes
0answers
29 views

Source for learning Functional derivative , Gateaux derivative

I have started doing research in optimal control theory. My area of research is PDE constrained optimization. I am facing significant difficulty in deriving the Gateaux Derivative of functionals ...
4
votes
0answers
58 views

From infinite dimensional function space to n-dimensional real space

I am an engineering student who works in the field of optimal control. Problems in this field are typically framed in infinite dimensional Sobolev space, $W^{k,p}(\Omega)$, as: \begin{equation} \...
0
votes
1answer
16 views

Linear quadratic regulator equivalent formulations?

I don't see why the following three forms of the LQR optimal control problem are equivalent: For $\begin{cases} x'=Ax+Bu \\ x(t_0)=x_0\end{cases}$ find $$\min_{u\in L^2(t_0,T; \mathbb{R}^m)} J(u)=\...
1
vote
0answers
45 views

Solution for first order ODE with discontinuous right hand side

I'm now approaching for the first time to first order differential equations with discontinuous right hand side. Let $A\subset \mathbb{R}\times\mathbb{R}^n$ and $f=f(t, x):A\to \mathbb{R}^n$. Fix $(...
0
votes
0answers
34 views

How to minimize the next functional using the Pontryagin Maximum Principle?

Best regards, I am asked to minimize the next functional $T(v)=\int_{A}^{B}\frac{dx}{v(x)}$, with $v\neq 0\text{ and } v\in V$, where $V=\{v\in C([A,B]):v(A)=v(B)=0\}$. If we assume that there ...
0
votes
1answer
35 views

A simple quadratic optimizer for only constraints on input

I'm going to implement an quadratic optimizer with C for embedded systems. I will do that because I need speed. But I have some trouble to find a quadratic optimizer for C that works with embedded ...
0
votes
0answers
41 views

Software recommendation needed for optimal control

My subject is optimal control of PDE and I want to put optimal control problems in Matlab to be solved but I don't know how to do this. I would be grateful if you could give me some references in that ...
0
votes
2answers
60 views

Is there any rule of thumb when it comes to selecting control/predict horizon for MPC?

I have a simple question: Is there any rule of thumb when it comes to selecting control/predict horizon for MPC? Normaly I set control and predict horizon equals, but I have heard that's not good ...
0
votes
0answers
16 views

Value iteration method of optimal stopping

In Lawler's introdcution to Stochastic process p89~93, the value iteration method is given for homogeneous Markov chain: $u_1(x)$ equal to the payoff function $f(x)$ if $x$ is an absorbing state and ...
1
vote
0answers
45 views

Optimal control problem with a path constraint which involves controls at two distinct time points

I am faced with an optimal control problem in continuous time which includes a path constraint which involves controls at two distinct points in time. I do not know how to approach this problem. I do ...
1
vote
1answer
24 views

Unit of time and normalization of time preference rates

Consider an infinite horizon cake eating differential game described by \begin{align} &\max_{u_1(t)} \int_0^\infty{e^{-r_1 t}\ln(u_1(t))dt}\\ &\max_{u_2(t)} \int_0^\infty{e^{-r_2 t}\ln(u_2(t))...
1
vote
1answer
50 views

Numerical solution of non-linear first order partial differential equation (HJB)

I am trying to solve a simple optimal control problem using the Hamilton-Jacobi-Bellman equation, numerically in Python. This is proving to be rather difficult as I end up having to solve the ...
0
votes
0answers
71 views

Formulation for calculus of variation with state-space constraint

I'm stuck on this question, let $B = \{x\in \mathbb{R}^n:|x|\leq 1\}$ be the unit ball in $\mathbb{R}^n$, consider the following minimizing problem $$ \inf_{x(\cdot) \in \mathcal{A}} \int_0^\infty e^{-...
0
votes
1answer
36 views

Questions about LQG with full information

I have implemented LQG in MATLAB software. But, now I do not know how to determine the value of optimal cost. Each way of calculating cost, returns a different value. Which one should I trust to ...
0
votes
0answers
32 views

The Riccati equation and its asymptotic behavior

Consider matrices $A\in\mathbb{R}^{n\times n},B\in\mathbb{R}^{n\times m}$, a positive semidefinite symmetric matrix $Q\in\mathbb{R}^{n\times n}$ and a positive definite symmetric matrix $R\in\mathbb{R}...
1
vote
1answer
57 views

Optimal control problem with fixed endpoint; what I am doing wrong?

I am trying to solve the following problem (I choose a numerical example). The state $x$ is governed by a differential equation linear in $x\in [0,1]$. Two variables, $\theta \in[0,1]$ and $\rho\in[0,...
0
votes
0answers
32 views

Time-Discretize a Linear Quadratic OCP (Bolza Function)

I'm trying to formulate an optimal control problem based off of this given Minimum-Time Cost Function: $$J(t_f)=\frac{1}{2}[x(t_f)-x_{des}(t_f)]^TP_f[x(t_f)-x_{des}(t_f)] + \frac{1}{2}\int_{t_0}^{t_f}(...
1
vote
0answers
102 views

Bellman's Principle of Optimality

I'm currently reading Pham's Continuous-time Stochastic Control and Optimization with Financial Applications however I'm slightly confused with the way the Dynamic Programming Principle is presented. ...
0
votes
1answer
29 views

How to find a transformation matrix which will make the system a chain of integrators?

Consider a system of the form $$\dot{x}(t)=Ax(t)+Bu(t)+\phi(t)+D(t)$$ I have $$\dot{x}(t)=\begin{bmatrix} -p_1 &G_b & 0 & 0 &0 \\ 0& -p_2 & p_3 & 0 & 0\\ 0& ...
0
votes
0answers
28 views

How to solve Heat equation by Riccati equation ?

I turned the heat equation into a product of two linear differential equations and I think of using Adomian decomposition method for solving it ? it is possible ?
1
vote
1answer
47 views

What is the cost function (performance index) for 2nd order ODE systems?

I have the following 2nd order linear system with appropriate initial conditions. $$\textbf{X}''(t)+\textbf{A}(t)\textbf{X}'(t)+\textbf{B}(t)\textbf{X}(t)=\textbf{F}(t)+\textbf{C}(t)\textbf{U}(t)$$ $\...
1
vote
1answer
56 views

How would I compute this matrix in Matlab?

I'm trying to compute the H$_\infty$ norm of a matrix using the paper: Bruinsma, N. A.; Steinbuch, M., A fast algorithm to compute the $H{\infty}$-norm of a transfer function matrix, Syst. Control ...
1
vote
0answers
28 views

Guess and verify: verification theorem for Hamilton-Jacobi-Bellman equation

Let $t \in [0,\infty)$ denote time, $x(t) \in X \subset \mathbb R_+$ the state and $u(t) \in \mathbb R_+$ the control. Consider the following optimal control Problem \begin{align} &V(x_0) = \max_{...
0
votes
0answers
27 views

How to design a Robust observer for a 2D system

Consider the second order system given by $\dot{x}=Ax+Bw(t)$, where $x\in\mathbb{R}^2$, $$A = \begin{bmatrix} {0},{6}\\ {-1} {-6} \end{bmatrix}, \quad B = \begin{bmatrix} {0}\\{1}\end{bmatrix}...
0
votes
0answers
52 views

Dynamic programming's principle of optimality as an abstract construct

In dynamic programming, the principle of optimality (refer to Bertsekas's Optimal Control, volume 1, page 18) is a statement that says: For any optimal policy $\pi$ , we always have a suboptimal ...
1
vote
0answers
26 views

Price of a stochastic game between an agent and the market

In the article Pricing via utility maximization and entropy from Richard Rouge and Nicole El Karoui, they define the value function of the optimization problem as \begin{align} V(x,C) = \dfrac{1}{\...
0
votes
0answers
22 views

linearizing dynamics about non fixed point for LQR implementation.

I am trying to implement LQR control for the cart pole system. I am curious if I can maintain a constant non-zero pole angle. So, I need to linearize my dynamics about my goal state. I know we can use ...
0
votes
0answers
34 views

difference between HJ and HJB equation

This question might be very simple: what is the difference between Hamilton-Jacobi (HJ) equation and Hamilton-Jacobi-Bellman (HJB) equation, especially in optimal control theory? My personal ...
1
vote
0answers
24 views

Equivalent formulations of stochastic HJB equation

I have some trouble understanding stochastic HJB equations. There are basically two forms of this equation that I have encountered in books, lecture notes etc... (one-dimensional case) 1) $rv(x)=\pi (...
0
votes
1answer
51 views

How can I add find the gain from root locus and poles?

I try to find the P-gain from a root locus plot where I know the poles. Assume that we got a reference model: $$G(s) = \frac{\omega_n^2}{s^2 + 2\zeta \omega_n s + \omega_n^2 }$$ Where $\zeta$ and $\...
0
votes
1answer
32 views

Injectivity trajectory to set singular control to zero

Consider a control system of the form $$ \dot x(t) = X(x(t)) + u(t)\, Y(x(t)) \qquad \qquad (*) $$ where $X,Y$ are two smooth vector fields, and $u$ is the (bounded measurable) control function. ...