How can two dependent variables have zero covariance? The following text is from page 59 of chapter 3 of the book Deep Learning by Ian Goodfellow et al.:

Independence is a stronger requirement than zero covariance, because independence also excludes nonlinear relationships. It is possible for two variables to be dependent but have zero covariance.


For example, suppose we ﬁrst sample a real number $x$ from a uniform distribution over the interval [−1,1]. We next sample a random variable $s$. With probability $\frac{1}{2}$, we choose the value of $s$ to be 1. Otherwise, we choose the value of $s$ to be −1. We can then generate a random variable $y$ by assigning $y=sx$. Clearly, $x$ and $y$ are not independent, because $x$ completely determines the magnitude of $y$. However, $Cov(x, y) = 0$.

What is the difference between sampling a real number and sampling a random variable?
How can I mathematically show that $Cov(x,y)=0$?
 A: It means the same thing. It's just talking about sampling a number from the uniform distribution, call it X, on [-1,1], i.e. from $X\sim Unif[-1,1]$.
To calculate the covariance, you might note the formula $Cov(X,Y)=E(XY)-E(X)E(Y)$. It is known that E(X)=0, beacause X is uniform on [-1,1], so pick the middle value. $E(Y)=E(SX)=E(S)E(X)$ (beacuse they are independent), and their individual expectations are 0, so E(Y)=0. We need to find E(XY) now. Note that $E(XY)=E(SX^2)=E(S)E(X^2)$ again because S and X are independent. So we get $0\cdot E(X^2)=0$. Thus $Cov(X,Y)=0$.
To show that X and Y are dependent events, note that the value of Y is half-determined by the value of X. So Y "depends" on X. To show mathematically that they are dependent, you must find sets A and B such that $X\in A$ and $Y\in B$ are events, where $P(Y\in B|X\in A)\ne P(Y\in B)$. Choose $A=[.25,.75],B=[.25,.75]$. Then $P(Y\in B)=\frac 1 4$ (this is because Y is also uniformly distributed on [-1,1]). But $P(Y\in B|X\in A)=\frac 1 2$. This is because given that $X\in[.25,.75]$, there is equal chance that $Y\in[.25,.75]$ or $Y\in[-.75,-.25]$. Thus we have found an events such that $P(Y\in B|X\in A)\ne P(Y\in B)$, and the random variables are dependent.
Two variables are independent if $P(X\in A\cap Y\in B)=P(X\in A)P(Y\in B)$ for all possible combinations of sets A, B such that $X\in A$ and $Y\in B$. Equivalently, that $P(X\in A|Y\in B)=P(X\in A)$ for all possible combinations of events. This is obtained by dividing the first equation by $P(Y\in B)$.
A: Covariance is a measure of the linear dependence of two random variables. $Cov(x,y)>0$ means something like the following: if I observe a relatively large $x$, I should also expect to observe a relatively large $y$. However, in the example you gave, if you observe a large $x$, you are equally likely to observe a large $y$ (if $s=1$) as you are to observe a small $y$ (if $s=-1$)! Hence $Cov(x,y)=0$, even though $y$ clearly depends on $x$.
To show $Cov(x,y)=0$ in your problem:
\begin{align}
Cov(x,y) &= Cov(x,sx)\\
&= \mathbb E[x^2 s] - \mathbb E[x]\mathbb E[s x] \\
&= \mathbb E[x^2] \mathbb [s]\\
&= 0
\end{align}
Where we used $\mathbb E[x]=\mathbb E[s]=0$, and the independence of $x$ and $s$, which implies $\mathbb E[x^2 s] = \mathbb E[x^2] \mathbb E[s]$.
In general, independent random variables necessarily have zero covariance, but random variables with zero covariance need not be independent as they may have higher moments which are nonzero.
