Assuming that the data set was $z$-standardized to zero mean and unit variance (also assuming that it does not contain constant vectors).

Then Pearson's r reduces to Covariance: $$\rho(X,Y) := \frac{Cov(X,Y)}{\sigma(X)\sigma(Y)} = Cov(X,Y)$$

Now I'm investigating the dissimilarity function $$d(X,Y):=\sqrt{1 - \rho(X,Y)}$$ which is the square root of a common transformation of $\rho$ for use as a dissimilarity function.

It can be shown that given above preconditions, it is in fact a linear multiple of Euclidean distance, and thus trivially metric:

$$\sqrt{\sum_i (x_i - y_i)^2} = \sqrt{\sum_i x_i^2+\sum_i y_i^2 - 2 \sum_i x_i\cdot y_i} \\ = \sqrt{n + n - 2n \cdot Cov(X,Y)} = \sqrt{2n} \cdot d(X,Y)$$ i.e. $$d(X,Y) = euclidean(X,Y) / \sqrt{2n}$$ Now I'm wondering if the properties of this function $d$ have been further explored. Is it metrical under a broader set of conditions than z-standardized data sets? Do you know related literature or proofs?

  • $\begingroup$ Hmm, the only thing which comes to my mind is that this means: changing the view into the data. It means that each observed person defines a dimension in the euclidean space and each item, measured at this person, defines a vector from the origin into that space. Assume only two items now: then d is the distance between the arrowheads of the vectors. The beforehand centering of the items means to look at the difference-vectors (from the vectors to the means-vectors for each item) and standardizing means adapting lengthes. I've fiddled a bit with this but have no yet ready article about this. $\endgroup$ – Gottfried Helms Jun 6 '14 at 10:20
  • $\begingroup$ Interesting idea. z-standardization is a linear transformation in the transposed space. If metric properties survive transposition, this would pretty much yield a proof. $\endgroup$ – Has QUIT--Anony-Mousse Jun 6 '14 at 10:58
  • $\begingroup$ @Anony-Mousse can you provide any insight on this related derivation? math.stackexchange.com/questions/2168995/… $\endgroup$ – slaw Mar 4 '17 at 20:20

Yes there are a lot of related papers that I came across couple of weeks ago, which talk about converting Pearson Correlation to Euclidean distance, when data is z-normalized.

Your question was back in 2013, I hope you are still interested:

  1. StatStream Statistical Monitoring of Thousands of Data streams in real time (First paper to proof the relation and find the formula).

  2. Exact Discovery of Time Series Motifs

  3. Logical Shapelets An Expressive Primitive for Time
  4. On Similarity-Based Queries for Time Series Data
  5. Time Series Join on Subsequence Correlation

And a lot more.

  • $\begingroup$ I'll be more than happy to discuss this more via emails or other forms of communication, as I'm working on a related topic to this. $\endgroup$ – Jarvis Aug 26 '15 at 4:51
  • $\begingroup$ For z-standardization I proved the relationship in the question already. I'm interested if the metric property exists with less assumptions, too. $\endgroup$ – Has QUIT--Anony-Mousse Aug 26 '15 at 5:34
  • $\begingroup$ I'm sorry, i don't quite understand your question, probably because i'm not a mathematician (i'm in the computer science field). Could you explain more? Thanks. $\endgroup$ – Jarvis Aug 26 '15 at 9:27
  • $\begingroup$ The question is if a similar result is true without z-standardization. $\endgroup$ – Has QUIT--Anony-Mousse Aug 26 '15 at 12:40
  • $\begingroup$ No, definitely not. Assume there are two time series: x and y of equal length m, and assume that y[i] = x[i]*4, i.e, each value of time series y is a multiple of the corresponding x[i]'s value. Then, the distance without z-normalziation will not be 0, but if you z-normalize x and y, then the distance will be 0. $\endgroup$ – Jarvis Aug 27 '15 at 22:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.