Finding the entropy of a random variable $Y$ with no given $P(Y)$? Say we are given two random variables $X$ and $Y$ that can take the values of ${a,b}$ only. X~Bern(p) is given and also the conditional probabilities $P(Y=a|X=a)$ and $P(Y=b|X=b)$. I am trying to find $H(Y)$ but only know so far $H(X,Y)$ and $H(Y|X)$. If the two random variables are independent (which is not stated in the problem), I could say that  $H(Y|X)=H(Y)$ but there it is not stated that they are independent. There are no other infos given, Y~Bern(p) is not given. I could use this:
$$H(Y,X) = H(Y) + H(X|Y)$$
But the problem is I don't know that $H(X|Y$) is either, neither $H(Y,X)$. Are there relationships that I am not aware of that would help me solve this?
 A: Forget the relationships, and work it out directly. In information theory, there are many formulae between the basic concepts but most are simply definition chases, with use of the law of total probability or other elementary ideas. They can help you out in some cases, but going back to the initial definitions will not slow you down too much.
Let $t:=\mathbb{P}(Y=a)$. Then $$\begin{align}H(Y)&=-\sum_x \mathbb{P}(Y=x)\log \mathbb{P}(Y=x)\\ &= -(\mathbb{P}(Y=a)\log \mathbb{P}(Y=a)+\mathbb{P}(Y=b)\log \mathbb{P}(Y=b))\\ &=-(t\log t+(1-t)\log (1-t))\end{align}$$
So we just need to work out what $t$ is, and then we can plug it in. You might notice that $a,b$ are not really relevant: entropy does not take into account what the values are, only how likely they are. The expression above has a name, the binary entropy function (a function of $t\in [0,1]$), and it's the entropy of a Bernoulli random variable (which by convention only takes values $0$ or $1$).
$$ t=\mathbb{P}(X=a)\mathbb{P}(Y=a|X=a)+\mathbb{P}(X=b)\mathbb{P}(Y=a|X=b)$$
This is the law of total expectation. You already know $\mathbb{P}(X=a)=p$ and $\mathbb{P}(X=b)=1-p$, and are given the value of $\mathbb{P}(Y=a|X=a)$.
That only leaves us to work out $\mathbb{P}(Y=a|X=b)$, but $Y=a$ is the same thing as $Y\neq b$ (as $Y$ can only take the values $a,b$), so $\mathbb{P}(Y=a|X=b)=1-\mathbb{P}(Y=b|X=b)$ and you are given $\mathbb{P}(Y=b|X=b)$ as well.
