4
$\begingroup$

Assuming that the data set was $z$-standardized to zero mean and unit variance (also assuming that it does not contain constant vectors).

Then Pearson's r reduces to Covariance: $$\rho(X,Y) := \frac{Cov(X,Y)}{\sigma(X)\sigma(Y)} = Cov(X,Y)$$

Now I'm investigating the dissimilarity function $$d(X,Y):=\sqrt{1 - \rho(X,Y)}$$ which is the square root of a common transformation of $\rho$ for use as a dissimilarity function.

It can be shown that given above preconditions, it is in fact a linear multiple of Euclidean distance, and thus trivially metric:

$$\sqrt{\sum_i (x_i - y_i)^2} = \sqrt{\sum_i x_i^2+\sum_i y_i^2 - 2 \sum_i x_i\cdot y_i} \\ = \sqrt{n + n - 2n \cdot Cov(X,Y)} = \sqrt{2n} \cdot d(X,Y)$$ i.e. $$d(X,Y) = euclidean(X,Y) / \sqrt{2n}$$ Now I'm wondering if the properties of this function $d$ have been further explored. Is it metrical under a broader set of conditions than z-standardized data sets? Do you know related literature or proofs?

$\endgroup$
3
  • $\begingroup$ Hmm, the only thing which comes to my mind is that this means: changing the view into the data. It means that each observed person defines a dimension in the euclidean space and each item, measured at this person, defines a vector from the origin into that space. Assume only two items now: then d is the distance between the arrowheads of the vectors. The beforehand centering of the items means to look at the difference-vectors (from the vectors to the means-vectors for each item) and standardizing means adapting lengthes. I've fiddled a bit with this but have no yet ready article about this. $\endgroup$
    – Gottfried Helms
    Jun 6, 2014 at 10:20
  • $\begingroup$ Interesting idea. z-standardization is a linear transformation in the transposed space. If metric properties survive transposition, this would pretty much yield a proof. $\endgroup$
    – Has QUIT--Anony-Mousse
    Jun 6, 2014 at 10:58
  • $\begingroup$ @Anony-Mousse can you provide any insight on this related derivation? math.stackexchange.com/questions/2168995/… $\endgroup$
    – slaw
    Mar 4, 2017 at 20:20

1 Answer 1

Reset to default
2
$\begingroup$

Yes there are a lot of related papers that I came across couple of weeks ago, which talk about converting Pearson Correlation to Euclidean distance, when data is z-normalized.

Your question was back in 2013, I hope you are still interested:

  1. StatStream Statistical Monitoring of Thousands of Data streams in real time (First paper to proof the relation and find the formula).

  2. Exact Discovery of Time Series Motifs

  3. Logical Shapelets An Expressive Primitive for Time
  4. On Similarity-Based Queries for Time Series Data
  5. Time Series Join on Subsequence Correlation

And a lot more.

$\endgroup$
8
  • $\begingroup$ I'll be more than happy to discuss this more via emails or other forms of communication, as I'm working on a related topic to this. $\endgroup$
    – Jarvis
    Aug 26, 2015 at 4:51
  • $\begingroup$ For z-standardization I proved the relationship in the question already. I'm interested if the metric property exists with less assumptions, too. $\endgroup$
    – Has QUIT--Anony-Mousse
    Aug 26, 2015 at 5:34
  • $\begingroup$ I'm sorry, i don't quite understand your question, probably because i'm not a mathematician (i'm in the computer science field). Could you explain more? Thanks. $\endgroup$
    – Jarvis
    Aug 26, 2015 at 9:27
  • $\begingroup$ The question is if a similar result is true without z-standardization. $\endgroup$
    – Has QUIT--Anony-Mousse
    Aug 26, 2015 at 12:40
  • $\begingroup$ No, definitely not. Assume there are two time series: x and y of equal length m, and assume that y[i] = x[i]*4, i.e, each value of time series y is a multiple of the corresponding x[i]'s value. Then, the distance without z-normalziation will not be 0, but if you z-normalize x and y, then the distance will be 0. $\endgroup$
    – Jarvis
    Aug 27, 2015 at 22:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .