1
vote
0answers
24 views

How to solve Bellman's optimal equation from the first principle

How to solve the following set (finite) of equations $$ v_*(s) = \max_{a\in A(s)} \sum_{s'} p(s'|s,a) [r(s,a,s') + \gamma v_*(s')]$$ $p$ and $r$ functions are given.
0
votes
1answer
44 views

it it possible to solve these equation for their root.

I am trying to solve an equations such as the roots of $$k*x(11*x + 1) + d*x(11x + 1)$$ has to match the roots of this function $$x^2 + 0.1x + 6 + k*x(11*x + 1) + d*x(11x + 1)$$, where I have to ...
2
votes
0answers
69 views

How to use mathematically the I and D of a PID controller

I am trying to mathmatically understand how the P, I and D parameters work on a system, quite having a hard time doing so. I've only been able to show that the Steady State Error (SSE) never ...
0
votes
1answer
36 views

Is this a discrete time Lyapunov function?

I have an algorithm to optimize a process. It is a discrete time algorithm. Every iteration of this algorithm changes the state of the process. I found a function, say $f(s)$, where $s$ is the state ...
0
votes
1answer
23 views

making a function non-linear using a Lagrangian function

How Is this formula a Lagrangian function ? And how can a non-linear element be added to a function using this "Lagrangian function" This is where i got this In order to improve the performance ...
3
votes
2answers
78 views

significance of zeros of a transfer function?

In control theory, the poles of a transfer function give information about the stability and behavior of a system. I'm not sure and can't find anywhere what the significance of the zeros of a ...
0
votes
1answer
22 views

Literature study for Optimal Estimation Theory

It seems Optimal Estimation/Control Theory requires a lot more than undergraduate maths. Any good book that would help me get started? I have so far referred the following books but found them quite ...
1
vote
0answers
44 views

analytic solution to structured algebraic Riccati equation

In solving a model I have written down for a research paper, I am left with two Algebraic Riccati Equations, that is I need to solve for $X$ in the equation \begin{align*} X = A^\top (X + XB(R + ...
2
votes
0answers
13 views

Controlled system question optimisation

Consider the controlled system $x_{t+1} = x_t + u_t + 3\epsilon_{t+1}$, where the $\epsilon_t$ are independent $N(0,1)$ variables. The instantaneous cost at time t is $x_t^2 + 2u_t^2$. Assuming that ...
1
vote
0answers
50 views

Continuous time optimisation question- find value function and optimal control

If we have a continuous-time system with a scalar state variable, plant equation $\dot{x}= u$, and cost function $Q\int_o^h u^2 dt + x(h)^2$, then by writing the dynamic programming equation ...
0
votes
1answer
17 views

Determination of the modulus of continuity

I'm trying to prove the uniqueness of the viscosity solution of an Hamilton-Jacobi-Bellman equation. Thanks to a classical result, I'm left to check if it exist a modulus of continuity $\omega_1$ --- ...
2
votes
0answers
70 views

Cellular Automata Method to Reach Equilibrium on a Network of Numbers

Here's a puzzle that I'm curious if anyone can attack with a simple method without resorting to simulation. I can tell you the answer (well, an answer) from running some programs. But I'm writing a ...
3
votes
1answer
71 views

Optimal consumption policy

I start with an initial capital C and at the beginning of day $n=1,...,N$ I observe the random variable $X_n$, where $\mathbb E X_n=\mu_n$. The $X_n$ are independent. I also choose $c_n$ on day $n$, ...
3
votes
0answers
76 views

Another machine problem

This is linked to my previous question Discounted optimization problem I have difficulties again finding a formula for $F(x)$. We consider a machine at time $t$ which is in state $x$. The machine ...
2
votes
2answers
36 views

Having trouble forming the initial matrices for a positioning problem

The question asks me to solve the positioning problem where: $$ \dot{x_1} = x_2 $$ $$ \dot{x_2} = u_1 \in U_{bb} $$ $$ x_1(0) = - \text{X} (<0) $$ $$ x_2 (0) = 0 $$ $$ x_1(t_1) = 0$$ $$ x_2(t_1) = ...
0
votes
1answer
68 views

online learning to maximize profit

I have a software which takes input as investment and gives the output as return on a particular stock. Now profit metric $x_i$ is defined as the ratio of return $g_i$ to maximum possible return ...
2
votes
1answer
60 views

On impulsive optimal control with functions of not bounded variation

I have the following optimal control problem $$ J=\int_0^TF(t,y_1(t),y_2(t))dt \to \min, $$ subject to \begin{align} &\dot y_1(t) = f(t,y_1(t),y_2(t)) + g(t)\nu(t),\\ &\dot y_2(t) = ...
0
votes
1answer
27 views

Minimization problem

For Which positive value(s) of $x$ the following function is most minimum $f(x) = x^2 + ax +c$ [ where $a ,c > 0$ ] [note : I know there is no positive $x$ for which $f(x)$ is minimum but I ...
6
votes
2answers
312 views

How can I, as a future mathematician, contribute most to Smart Grid research?

After I've finished my Master's degree in mathematics, I too want to use my powers for good. One endeavour I consider good is the pursuit of the design and implementation of a Smart Grid which will, ...
2
votes
0answers
61 views

Find u that minimizes the integral mean

How do I find $u : [0,\infty) \to \mathbb{R}^m$ that minimizes \begin{equation} J(u(\cdot)) = \lim_{t \to \infty} \frac{1}{t} \int_0^t L(x(\tau),u(\tau)) d \tau, \end{equation} subject to ...
0
votes
1answer
108 views

Positioning problem optimal control

Consider the positioning problem: $\dot x_1 = x_2$, $\dot x_2 = u$ with $x_1(0)=0,x_2(0)=X, X>0$. show that the bang-bang control switch can be employed to steer the system to the origin. Find the ...
0
votes
1answer
85 views

Understanding the Hamiltonian function

Based on this function: $$\text{max} \int_0^2(-2tx-u^2) \, dt$$ We know that $$(1) \;-1 \leq u \leq 1, \; \; \; (2) \; \dot{x}=2u, \; \; \; (3) \; x(0)=1, \; \; \; \text{x(2) is free}$$ I can ...
3
votes
1answer
132 views

minimization problem on differential equations - optimal control

I am trying to minimize an time-integral of a linear function with respect to differential equations. The problem is formally defined as follows: Given $\lambda< \mu_1, \mu_2$ fixed ...
2
votes
0answers
57 views

How verification argument really works?

Let $C(u,s)$ be cost functional for an admissible control $u$ with initial state of the system being $s$. Our aim is the solution of the following problem: $$\inf_u E(C(u,s))$$ We defined the value ...
3
votes
2answers
867 views

Solution of a Sylvester equation?

I'd like to solve $AX -BX + XC = D$, for the matrix $X$, where all matrices have real entries and $X$ is a rectangular matrix, while $B$ and $C$ are symmetric matrices and $A$ is formed by an outer ...
3
votes
1answer
129 views

Verification for maximum principle

Given optimal control problem $$ \dot x = f(t,x(t),u(t)), \quad x(0) = x_0,\\ J(u) = \int_0^T f^0(t,x(t),u(t))dt \to \min, $$ we can apply Pontryagin's maximum principle to get a necessary condition ...
2
votes
1answer
87 views

How can I solve this control problem?

Consider this control problem in continuous time, known as Representative Agent Model in macroeconomics: $$ \max_{c_t,t\ge 0}\int_{0}^{\infty}e^{-\rho t}\ln(c_t)\, \mathrm{d}t,~~~\rho\in (0,1) $$ ...
1
vote
1answer
91 views

2nd Order Optimal Control Problem

I'm working on a homework problem in optimal controls and my plant model is described as: $$\ddot{x}(t) = u(t)$$ The performance index (cost function) is described by: $$J = 1/2\int_0^5u^{2}(t)dt\,$$ ...
7
votes
2answers
639 views

Difference between Bellman and Pontryagin dynamic optimization?

Can someone please explain the difference between dynamic optimization via the Bellman equation and dynamic optimization via Pontryagin's maximization principle? Thanks