Linear regression with normalized variables Suppose I have two variables X and Y such that mean(X) = 0 = mean(Y) and sd(X) = 1 = sd(Y). The slope of the linear regression line for Y vs X is cov(X,Y)/var(X) = corr(X,Y) since X and Y are normalized. Suppose corr(X,Y) < 1. By the same token, the slope of the linear regression line for X vs Y is corr(Y,X) = corr(X,Y) < 1. However, the slope of the best fit line for X vs Y should be the reciprocal of the slope of the best fit line for Y vs X (by reflecting and rotating the co-ordindate axes, the best fit line should not change). So the slope of the best fit line of X vs Y should be greater than 1. What am I missing here?
 A: The slope of the "inverse regression" is not the reciprocal. That is, if 
$$Y = b_0 + b_1 X + e$$
and 
$$X = a_0 + a_1 Y + \epsilon$$
then $a_1 \ne 1/b_1$ (except in the trivial case). You have given a nice reason why not!
A: What you said is correct. In the more general case, if $X$ and $Y$ are normalized variables, and $X$ has a standard deviation of $\sigma_{X}$ and $Y$ has a standard deviation of $\sigma_{Y}$, then the slope of the regression line for $Y$ vs. $X$ is given by $a_{XY}=\frac{Cov(X,Y)}{\sigma_{X}^{2}} = Corr(X,Y)\frac{\sigma_{Y}}{\sigma_{X}}$. Now, if we plot $Y$ against $X$ we obtain $a_{YX}= \frac{Cov(Y,X)}{\sigma_{X}^{2}} = Corr(Y,X)\frac{\sigma_{X}}{\sigma_{Y}}$. Since $Corr(Y,X)=Corr(X,Y)$ and $Cov(X,Y)=Cov(Y,X)$, we see that $a_{YX}=a_{XY}\frac{\sigma_{X}}{\sigma_{Y}^{2}}$. In particular, if $\sigma_{X}=\sigma_{Y}=1$ (or, more generally, if $\sigma_{X}=\sigma_{Y}^{2}$), then $a_{YX}=a_{XY}$. 
I believe your original question is why would reflecting the regression line about $Y=X$ would not also be the least squares regression. The answer is that the line obtained by rotating your whole graph by 90 degrees counter-clockwise (and thus switching the $X$ and $Y$) axes will minimize the horizontal least squares, not the vertical ones, i.e. what needs to minimized is now $\Sigma_{j} |a_{YX}X-Y|^{2}$ instead of $\Sigma_{j} |a_{XY}Y-X|^{2}$ (this is still assuming that both $X$ and $Y$ have mean zero).  In fact, the formula for both $a_{XY}$ and $a_{YX}$ can be derived from these by differentiating with respect to $a_{XY}$ and $a_{YX}$ respectively. 
