If we have two separate probability distributions P(x) and Q(x) over the same random variable x, we can measure how diﬀerent these two distributions are using the Kullback-Leibler (KL) divergence...
The above statement is from Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville and I have the following question:
As far as I have understood, a random variable is defined considering a specific probability distribution in mind, it takes the value of a random outcome in that distribution. Perhaps I'm wrong in my understanding. My question is:
How can you have two separate probability distributions on the same random variable?
Kindly help me resolve this confusion. Thanks!