1
$\begingroup$

Simply:

What is the intuition you get about the data if you are given standard deviation?

More detailed:

It is easy to imagine some information about the data if you are being told for example mean or median, etc. Also if you are told that some quantity is for example in range $5 \pm 0.001$, you again have idea about the value and also about its range. But can you make similar intuition about the data if you are given standard deviation (or maybe another more feasible quantity)?

I understand that this is perhaps dependent on probability distribution, but that is unfortunately rarely discussed in practice, data are usually measured, put into table and some basic statistics are mentioned. Let's say for example you are software developer and you need to measure latency of the system over time and present this to the management, they are usually not interested in probability distributions (although I feel that this is wrong and one should know or assume some probability distribution whenever working with the data...).

Note:

Please note that this is not a question about how to calculate standard deviation or why specific formula for standard deviation has been chosen, as some another questions here already address this. This question is strictly about having practical intuition when working with the data (providing or interpreting).

Feel free to provide any examples you like to demonstrate the answer, I did not want to limit this question by focusing on too specific situation.

$\endgroup$
  • 1
    $\begingroup$ Standard deviation tells you how far from the mean you could potentially end up in your measurements. For a normal distribution, you typically learn in more practical statistics courses that "mean $\pm$ 1 std. dev. = 67%", "mean $\pm$ 2 std. dev. = 95%", and "mean $\pm$ 3 std. dev. = 99.7%", giving you some handy confidence intervals off the top of your head. More info here: en.wikipedia.org/wiki/Standard_deviation $\endgroup$ – A.Sh Apr 10 '16 at 10:51
  • $\begingroup$ Thanks, that looks good, didn't know about the 68-95-99.7 rule. It seems like exactly what i was looking for. Only bad is that i have to assume the distribution is normal. $\endgroup$ – Sil Apr 10 '16 at 12:29
1
$\begingroup$

For general distributions, Chebyshev's inequality is applicable https://en.wikipedia.org/wiki/Chebyshev%27s_inequality.

It says that $1-\frac{1}{k^2}$ of the data falls within $k$ standard deviations of the mean. (E.g. $\frac34$ of the data falls within $2$ standards deviations of the mean.)

$\endgroup$
1
$\begingroup$

The comment and @bubba 's answer offer useful technical information.

For your use case

Let's say for example you are software developer and you need to measure latency of the system over time and present this to the management

I'd recommend some care in collecting measurements. The latency is probably highly dependent on system load, perhaps therefore on time of day, so you can't simply average values in the time series and get a useful number, even if you qualify it with a "plus or minus" using the standard deviation. You should try to get many measurements for each hour of the day (say). In any particular set of hourly data the values are likely to be more or less normally distributed, so the standard deviation is a good measure of the uncertainty of the average.

Management might learn more from a graph than from numbers. Perhaps a plot with error bars, like the one at https://stackoverflow.com/questions/12957582/matplotlib-plot-yerr-xerr-as-shaded-region-rather-than-error-bars . Excel knows how to build those.

$\endgroup$
1
$\begingroup$

For any distribution that's roughly normal, you can get a good feel from the 68, 95, 99.7 rule mentioned in the comment.

More generally, for arbitrary distributions, the standard deviation measures the variability of the data. More specifically, it measures the average distance of values from the mean. A small standard deviation means that most values are close to the mean.

If you know any physics or mechanical engineering, standard deviation is closely related to moments of inertia. Imagine the graph of the probability density function made from a thin sheet of metal. The standard deviation measures the moment of inertia of this piece of metal about a vertical axis through the mean. So, if the standard deviation is small, he moment of inertia will be small, which means the shape will be easy to spin. An object is easy to spin when most of its mass is located close to the rotation axis.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.