I'm stuck with understanding pretty simple expression and would appreciate some help on this. The most interesting part for algorithms, it's way how we can come here.
Using the original resources from Andrej Karpathy blog about Policy Gradient. Everything is clear with Monte Carlo credit assignments and supervised algorithms vs reinforcement. We have next expression, how we came up this optimization objective and gradient for it (images from another resources):
1) I'm familiar with derivation I think, but what was a point for taking log in this case? It's called likehood ratio trick sometimes and also explained here (where I still cannot get it). What is the point of using it here?
2) Can somebody show few Very simple examples of using it with numbers and how it works? Is there anything else about math I need to find or this could exist on Khan Academy?
Please consider answering above two points. I don't need to find derivative of
softmax and complicated output. I would appreciate some new explanation (different from articles above). And let's say that action space it's continues value and probability of taking action it's liner activation within very simple example.