# Tag Info

20

There are, in fact, two different formulas for standard deviation here: The population standard deviation $\sigma$ and the sample standard deviation $s$. If $x_1, x_2, \ldots, x_N$ denote all $N$ values from a population, then the (population) standard deviation is $$\sigma = \sqrt{\frac{1}{N} \sum_{i=1}^N (x_i - \mu)^2},$$ where $\mu$ is the mean of the ...

17

Your guess is correct: least absolute deviations was the method tried first historically. The first to use it were astronomers who were attempting to combine observations subject to error. Boscovitch in 1755 published this method and a geometric solution. It was used later by Laplace in a 1789 work on geodesy. Laplace formulated the problem more ...

9

A standard procedure (frequently -and loosely- called 'bayesian average') is to make a weighted average between the individual rating and the 'a priori' rating: $R_a = W \; R + (1 - W ) \; R_0$ where $R_a =$ averaged ('bayesian') rating $R =$ individual rating: average rating for this item. $R_0 =$ a priori rating: global average rating, for all ...

9

This is false in general. In fact, it is true if and only if $X=aY+b$ (almost surely) for some fixed constants $a \geq 0$ and $b \in \mathbb{R}$. That is $X$ and $Y$ must be positively linearly related for this to hold. Wikipedia also has a decent page on this. For a counterexample to your statement, consider any two independent random variables $X$ and $Y$ ...

8

I think the easiest way to do this is with an orthogonality trick. I'll show how to incrementally compute the variance instead of the standard deviation. Let $X_1, X_2, ...$ be an iid sequence of random variables with $\bar X = n^{-1} \sum_{j = 1} ^ n X_j$ and $s^2_n$ defined similarly as the $n$'th sample variance (I use a denominator of $n-1$ instead of ...

7

There is. Your alternative formulation of taking the absolute values of the differences instead of squaring them is called the mean absolute deviation (or average absolute deviation). Both the mean absolute deviation and the standard deviation are used in practice, but much of the reason the standard deviation is more widely used is that it has nicer ...

7

They are not the same. If you have a bimodal distribution with two peaks and allow the spacing between them to vary, the standard deviation would increase as the distance between the peaks increases. However, the entropy $$H(f) = \int f(x) \log f(x) dx$$ doesn't care about where the peaks are, so the entropy would be the same.

6

The "degrees of freedom" explanation of using $n-1$ for the sample standard deviation is close to hand-waving. The use of $n$ in calculating population variance, and so population standard deviation, comes from the definition of variance for a set with a number of equally probable outcomes. It is consistent with the definition for discrete distributions ...

6

While cardinal has answered the question, I wanted to add a little something extra. The Cauchy-Schwarz inequality shows that $\left|\operatorname{Cov}(X,Y)\right|\leq \sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}$, with equality if and only if $X$ and $Y$ are linearly dependent. In general, you can take the ratio $\rho=\frac{\operatorname{Cov}(X,Y)}{ ... 6 You could consider each post to have a 'true' average rating which you zero in on as more users vote on it. If we consider that the votes on each post come from a set of all possible votes that could have been cast that have a 'true' mean$\mu$and standard deviation$\sigma$, then your average of the votes actually cast can be considered as an estimator of ... 6 Hint: Write, $$\tag{1}\textstyle P[\,X>31\,] =P\bigl[\,Z>{31-\mu\over\sigma}\,\bigr]=.2743\Rightarrow {31-\mu\over\sigma} = z_1$$ $$\tag{2}\textstyle P[\,X<39\,] =P\bigl[\,Z<{39-\mu\over\sigma}\,\bigr]=.9192\Rightarrow {39-\mu\over\sigma} =z_2 ,$$ where$Z$is the standard normal random variable. You can find the two values$z_1$... 6 Squaring is nicer than taking the absolute value, e.g. it is smooth. It also leads to a definition of variance which has nice mathematical properties, e.g. it is additive. But for me the theorem that really justifies using standard deviation over the mean absolute error is the central limit theorem. The central limit theorem is at work whenever we measure ... 5 If you look closely at the Wikipedia article that you quoted, you will see that you computed either what they call the "population standard deviation" or what they call the "standard deviation of the sample," and not what they call the "sample standard deviation!" To find the sample standard deviation, we need to divide$60$by$8$, not$9$, and then take ... 5 If you take your cues from the financial industry, you can use the coefficient of variation (CV), which is the standard deviation / mean. This formula is used to normalize the standard deviation so that it can be compared across various mean scales. As a rule of thumb, a CV >= 1 indicates a relatively high variation, while a CV < 1 can be considered ... 5 Such a formula cannot exist, since it would compute$E(XY)$only using (characteristics associated to) the separate distributions of$X$and$Y$.$E(XY)$depends, as you noted, on the joint distribution of$X$and$Y$. The missing figure you need to get a formula as you would like is the correlation coefficient. 5 For any two random variables: $$\text{Var}(X+Y) =\text{Var}(X)+\text{Var}(Y)+2\text{Cov}(X,Y).$$ If the variables are uncorrelated (that is,$\text{Cov}(X,Y)=0$), then $$\tag{1}\text{Var}(X+Y) =\text{Var}(X)+\text{Var}(Y).$$ In particular, if$X$and$Y$are independent, then equation$(1)$holds. In general $$\text{Var}\Bigl(\,\sum_{i=1}^n X_i\,\Bigr)= ... 4 v_n is going to be \sum_{i=1}^n (Y_i - \overline{Y}_n)^2 where \overline{Y}_n = \frac{1}{n} \sum_{i=1}^n Y_i. Note that by expanding out the square, v_n = \sum_{i=1}^n Y_i^2 - \frac{1}{n} \left(\sum_{i=1}^n Y_i\right)^2. In terms of m_k = \sum_{i=1}^k Y_i, we have$$v_n = \sum_{i=1}^n Y_i^2 - \frac{1}{n} m_n^2 = \sum_{i=1}^{n-1} Y_i^2 + Y_n^2 - ... 4 We will find the inflection points of the density function of a general one-variable normal. This can be done in the usual calculus way, by examining the second derivative of the density function. The density function$f(x)$of the general one variable normal is given by $$f(x)=\frac{1}{\sqrt{2\pi}\sigma}\exp((x-\mu)^2/2\sigma^2).$$ Differentiate. We get ... 4 Quantities are being squared, summed, and square rooted for the same reason that you do these things when finding the length of a vector. Just to pick a number, suppose that you have$10$data points. Then before you took any measurements, there were ten variables$X_1, X_2, \ldots X_{10}$. Once you take the measurements, you have a vector in ... 4 If you divide all the values by the standard deviation then you will then have a distribution with a standard deviation equal to$1$(and so a variance equal to$1^2 = 1$). The difference is that the mean is not$0$, unless it was originally. You seem to be confusing the variance with (half) the range. The range is the difference between the minimum ... 4 What they are talking about is the value of the standard deviation. When the value is assumed to be known you can divide the sample mean by it and if the samples have a normal distribution the sample mean minus the population mean divided by the "known" standard deviation divided by the square root of the sample size n has a standard normal distribution (a ... 4 Assume we are talking about a standard normal distribution with zero mean and unit variance. For the observer to be able to answer the question "is this sample from a standard normal distribution?" with a high probability of correctness, he needs to know the distribution of distributions from which the sample may have been generated. The probability that ... 4 The square is used to remove the effect of the sign of$x_i - \overline{x}$. Suppose your mean was 0, and you had measurements at -2 and +2. These would cancel, but squaring gets rid of that issue. Now, you might ask, "why not use absolute value?" Great question! The reason is that if we use absolute value, variances are no longer additive. In other words, ... 4 The probability that a random variable$Z$with standard normal distribution is less than$7.7$is, for all practical purposes, equal to$1$. We have $$\Pr(Z \gt 7.7)\approx 6.8\times 10^{-15}.$$ The probability that we are$7.7$or more standard deviations away from the mean (either direction allowed) is twice that. But twice utterly negligible is still ... 4 The expression in your title does not define a normal density unless you assume that$\sigma > 0$, and indeed most people would use$\sigma^2$in the normal density as you have indeed done in the text of your question. Provided that$\sigma > 0$, note that $$\frac{\gamma}{\sqrt{2\pi\sigma}}\exp\left(-\frac{\gamma^2}{\sigma}\frac{(x-\mu)^2}{2}\right) = ... 3 Posting as an answer in response to comments. Here's a way to compute the mean and standard deviation in one pass over the file. (Pseudocode.) n = r1 = r2 = 0; while (more_samples()) { s = next_sample(); n += 1; r1 += s; r2 += s*s; } mean = r1 / n; stddev = sqrt(r2/n - (mean * mean)); Essentially, you keep a running total of the sum of ... 3 Likelihood function is generally ratio of two densities. Since working with ratios is not convenient when taking max, one normally takes the logs. This is most effective when densities involve exponential functions and is extremely convenient in the case of normal (Gaussian) densities. Taking logs does not always help. In fact if the underlying densities ... 3 Typically you have a collection of values of some variable. If the variable is human height, these values might be expressed in inches or centimetres; if it’s human weight, in pounds or kilograms. The standard deviation is a specific number of these units; roughly speaking, it’s the size of a typical deviation from the mean of these measurements. To compare ... 2 The Fibonacci numbers, the larger ones in particular, are very close to being given by the formula$$ F_n\approx\frac1{\sqrt5} \left(\frac{1+\sqrt5}2\right)^n $$(F_n is always the closest integer to the r.h.s.). In light of this I would also consider calculating, given input x, how far the ratio$$ ... 2 If$\sigma$is the (population) standard deviation then $$\sigma^2 = \frac{\sum (x_i - \mu)^2}{n} = \frac{\sum x_i^2}{n} - \mu^2$$ so using the subscripts$1$,$2$and$C$for the first part, second part and combination $$\sigma_C^2 = \frac{n_1 (\sigma_1^2 + \mu_1^2) + n_2 (\sigma_2^2 + \mu_2^2)}{n_1+n_2} - \mu_C^2$$ where$\mu_C = \frac{n_1 \mu_1 + ...

Only top voted, non community-wiki answers of a minimum length are eligible