# Cross-Entropy loss in Reinforcement Learning

In the context of supervised learning for classification using neural networks, when we are identifying the performance of an algorithm we can use cross-entropy loss, given by: $$L = -\sum_1^n log(\pi (f(x_i))_{y_i})$$ Where $x_i$ is a vector datapoint, $\pi$ is a softmax function, $f$ is our nerual network, and $y_i$ refers to the correct class for that data point. So the result is that we will get the the $log$ of the normalised network output for the correct class. The smaller L is the more accurate our model will be. So we can view this as a minimisation problem, where we just need to minimise $L$ with respect to the network weights. However, if we now look at how this equation is used in reinforcement learning, where now the network outputs represent an action which is chosen to interact with an environment instead of a class, we get something like this: $$L = -\sum_1^n r_i log(\pi (f(x_i))_{a_i})$$ Where $r_i$ is a scalar representing the reward at a time step, and $a_i$ is the corresponding chosen action. As we are now in the context of reinforcement learning $a_i$ is used because we don't know what the correct class action is, but by multiplying each network output by the reward, it has the effect where it pushes the gradient in right direction when we perform back propagation to our network.

Although this is how the differences in equations have been explained to me, there is one thing that doesn't quite add up. In the reinforcement learning version of the equation, $L$ now increases when the agent performs well, and decreases when the agent performes badly. Where a negative reward means punishment. Which makes me think this has become a maximisation problem, however, all the implementations I've seen still treat the equation as a minimisation problem. Why is that? Have I misunderstood the maths?