# Tagged Questions

24 views

### How to solve Bellman's optimal equation from the first principle

How to solve the following set (finite) of equations $$v_*(s) = \max_{a\in A(s)} \sum_{s'} p(s'|s,a) [r(s,a,s') + \gamma v_*(s')]$$ $p$ and $r$ functions are given.
44 views

### it it possible to solve these equation for their root.

I am trying to solve an equations such as the roots of $$k*x(11*x + 1) + d*x(11x + 1)$$ has to match the roots of this function $$x^2 + 0.1x + 6 + k*x(11*x + 1) + d*x(11x + 1)$$, where I have to ...
69 views

### How to use mathematically the I and D of a PID controller

I am trying to mathmatically understand how the P, I and D parameters work on a system, quite having a hard time doing so. I've only been able to show that the Steady State Error (SSE) never ...
36 views

### Is this a discrete time Lyapunov function?

I have an algorithm to optimize a process. It is a discrete time algorithm. Every iteration of this algorithm changes the state of the process. I found a function, say $f(s)$, where $s$ is the state ...
23 views

### making a function non-linear using a Lagrangian function

How Is this formula a Lagrangian function ? And how can a non-linear element be added to a function using this "Lagrangian function" This is where i got this In order to improve the performance ...
78 views

### significance of zeros of a transfer function?

In control theory, the poles of a transfer function give information about the stability and behavior of a system. I'm not sure and can't find anywhere what the significance of the zeros of a ...
22 views

### Literature study for Optimal Estimation Theory

It seems Optimal Estimation/Control Theory requires a lot more than undergraduate maths. Any good book that would help me get started? I have so far referred the following books but found them quite ...
44 views

### analytic solution to structured algebraic Riccati equation

In solving a model I have written down for a research paper, I am left with two Algebraic Riccati Equations, that is I need to solve for $X$ in the equation \begin{align*} X = A^\top (X + XB(R + ...
13 views

### Controlled system question optimisation

Consider the controlled system $x_{t+1} = x_t + u_t + 3\epsilon_{t+1}$, where the $\epsilon_t$ are independent $N(0,1)$ variables. The instantaneous cost at time t is $x_t^2 + 2u_t^2$. Assuming that ...
50 views

### Continuous time optimisation question- find value function and optimal control

If we have a continuous-time system with a scalar state variable, plant equation $\dot{x}= u$, and cost function $Q\int_o^h u^2 dt + x(h)^2$, then by writing the dynamic programming equation ...
17 views

### Determination of the modulus of continuity

I'm trying to prove the uniqueness of the viscosity solution of an Hamilton-Jacobi-Bellman equation. Thanks to a classical result, I'm left to check if it exist a modulus of continuity $\omega_1$ --- ...
70 views

### Cellular Automata Method to Reach Equilibrium on a Network of Numbers

Here's a puzzle that I'm curious if anyone can attack with a simple method without resorting to simulation. I can tell you the answer (well, an answer) from running some programs. But I'm writing a ...
71 views

### Optimal consumption policy

I start with an initial capital C and at the beginning of day $n=1,...,N$ I observe the random variable $X_n$, where $\mathbb E X_n=\mu_n$. The $X_n$ are independent. I also choose $c_n$ on day $n$, ...
76 views

### Another machine problem

This is linked to my previous question Discounted optimization problem I have difficulties again finding a formula for $F(x)$. We consider a machine at time $t$ which is in state $x$. The machine ...