2
$\begingroup$

I am sorry if this is too complicated a question to answer simply.

I am interested in understanding the mathematical intution behind why standard deviation and correlation are unitless when the metrics from which they are directly calculated (variance and covariance) both have units attached to them.

Its not really obvious as to why taking the square root of the variance produces the standard deviation and why is it unitless, and why does dividing the covariance by the product of the standard deviations always gives a number between $-1$ and $+1$ and why it is the correlation and why is it unitless.

I am unable to find any texts that offer any simple explanation for this. Hence I need some help understanding it as simply as possible.

$\endgroup$
  • $\begingroup$ Usually confidence limits are given in terms of standard deviations i.e. standard deviation is not unitless, it has the same units as the dataset. $\endgroup$ – acarturk Sep 10 at 11:59
  • $\begingroup$ Std isn't unitless. It has the same units as the random variable. Pearson correlation coefficient is unitless by definition: cov(X,Y) has units [X][Y], divided by stds of X and Y, it becomes unitless. The question why it is always between -1 and +1 can be answered with the help of Cauchy-Schwarz inequality. $\endgroup$ – Vasily Mitch Sep 10 at 12:01
4
$\begingroup$

Standard deviation isn't unitless. If I have some random variable measured in meters, and the standard deviation is $1$, then the same variable converted to feet will have stqandard deviation $3.28$. The standard deviation has the same unit as the variable, and will scale with them when you change units.

The correlation coefficient, on the other hand, is unitless. If you have two random variables measured in meters, and the correlation is $0.7$, then the correlation is still $0.7$ if you convert the samples to meters. Or even if you just convert one of the variables. It is unitless because you take the covariance and divide by the product of standard deviations. Scaling the value of the samples (by changing units) will shange the covariance in exactly the same way that it changes the product of the standard deviations, and the division makes the changes cancel out.

The correlation of two variables $X$ and $Y$ lie between $-1$ and $1$ because the covariance necessarily lies between $-\sigma_X\sigma_Y$ and $\sigma_X\sigma_Y$: If $Y = aX + b$ for $a, b\in \Bbb R, a\neq 0$, then $\operatorname{cov}(X, Y) = \pm \sigma_X\sigma_Y$ (depending on the sign of $a$), and any deviation from this will cause a covariance closer to $0$. However, the covariance isn't unitless, meaning a rescaling of one or two of the variables will result in a different covariance.

$\endgroup$
  • $\begingroup$ Related to this, the covariance defines an inner product between random variables. (Well, equivalence classes of random variables with finite second moment: $X$ and $X+1$ are indistinguishable w/r/t covariances, for instance.) In that case, the standard deviation defines the norm of a random variable. Hence the correlation coefficient $\text{cov}(X,Y)/(\sigma_X \sigma Y)$ can be interpreted as the cosine of the angle between $X$ and $Y$. $\endgroup$ – Semiclassical Sep 10 at 12:44
  • $\begingroup$ @Arthur So correlation can be thought of just as a ratio between covariance and the the product of the standard deviations. But can you tell me why divide by the product of the standard deviations ? I mean why not the summation or any other combination of the standard deviations ? $\endgroup$ – ng.newbie Sep 12 at 13:22
  • $\begingroup$ @Arthur Wait if $\operatorname{cov}(X, Y) = \pm \sigma_X\sigma_Y$ then shouldn't correlation always be $1$ ? I mean you are dividing $\pm \sigma_X\sigma_Y$ by $\pm \sigma_X\sigma_Y$. I don't understand this part of your answer. $\endgroup$ – ng.newbie Sep 12 at 13:26
  • $\begingroup$ @ng.newbie That's only if $Y = aX+b$ for fixed $a$ and $b$. The $a$ and $b$ have to be the same for every sample. For instance, if you throw a die, and $X$ is the result on the die, and $Y$ is two times the result. Then $X = \frac12Y$. In such a case you have "total" covariance (don't know if that's an established term), and a correlation of $1$ or $-1$ (you are always dividing by $\sigma_X\sigma_Y$, never $-\sigma_X\sigma_Y$). $\endgroup$ – Arthur Sep 12 at 14:22
  • $\begingroup$ @Arthur You didnt tell me why you divided by the product of the standard deviations. Why not the addition of them, what makes the product the correct answer ? $\endgroup$ – ng.newbie Sep 17 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.