What is the covariance between two random variables? What does the covariance of two random variables actually tell us? I've looked everywhere and I can't find a clear answer that I fully understand.
 A: It measures the linear association between the variables. In fact, if you divide the covariance by the product of the standard deviations, you get the correlation between the two variables. The difference is that the covariance depends on the units in which the variables are measured, whereas the correlation is a dimensionless number between $-1$ and 1. If you run the covariance on two variables measures in feet versus the same two variables measured in inches, the numbers will be different. This is not the case for the correlation between the variables.
To see why covariance measures linear association, look closely at the definition. For random variables $x$ and $y$, the covariance is $$E[(x-E[x])(y-E[y])]$$
where $E$ is the expectation.
Say you have a scatterplot of data $(x,y)$. Draw vertical and horizontal lines at the means $E[x]$ and $E[y]$, respectively. This splits your scatterplot into four quadrants. (Forget the $x$ and $y$ axes.) Then $x-E[x]$ and $y-E[y]$ have the same sign when $(x,y)$ is either in the first or third quadrants (bottom left or upper right). Therefore 
$$(x-E[x])(y-E[y])$$
is positive for data points in the first and third quadrants and negative for data points in the second and fourth quadrants. If the expected value of these products comes out to be positive, then "on the average" your data points lie in the first and third quadrants, which is a trend with a positive slope. If the expected value comes out to be negative, then "on the average" your data points lie in the second and fourth quadrants, which is a trend with a negative slope.
Note that this calculation really only measures a kind of linear association. It's measuring, roughly, whether the variables tend to move in the same or opposite directions. It is not measuring whether the variables have some relationship, period. For example, the data points might show a clear quadratic relationship by lying close to a parabola, but in this case the covariance could come out to be close to zero -- at least if the parabola has roughly the same number of points in all for quadrants. Just because the covariance is zero doesn't mean the variables aren't related. It just means that, on the average, there is no linear relationship.
