1
$\begingroup$

I have the following questions:

  1. Can I consider $G(f)$ to be a functional, where

    $$ G(f) = \int_{\mathcal{A} \times \mathcal{S}}\bigr[ R(s,a) +\gamma \int_{\mathcal{S}} \int_{\mathcal{A}}f(a',s')\pi(a'|s')P(s'|s,a) - f(s,a) \bigl]^2 \mu(s,a) $$ and where $\mu$ is a probability measure over $\mathcal{S}\times\mathcal{A}$, $\pi$ is a probability measure over $\mathcal{A}$, $P$ is a probability measure over $\mathcal{S}$, $R:\mathcal{S}\times\mathcal{A} \to \mathbb{R}$, $\gamma \in \mathbb{R}$ and $f:\mathcal{S}\times\mathcal{A}\to\mathbb{R}$?

  2. If $G$ is a functional, then, what is the analytic expression of $\nabla_{f} G(f)$? By intuition I think that should be: $$ \nabla_f G(f)\big|_{f = f_0} = h $$ where $$ h(s,a) = 2\bigr[ R(s,a) + \gamma \int_{\mathcal{S}} \int_{\mathcal{A}}f_0(a',s')\pi(a'|s')P(s'|s,a) - f_0(s,a)\bigl] \mu(s,a). $$

    Since I don't know almost anything about functional analysis, could you show to me the most crucial passages?

Thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.