2
$\begingroup$

EDIT: I'd like to reformulate my primary question.

I have a set of points in $\mathbb{R}^D$ and I reduce the dimensions of the points to some $\mathbb{R}^L$. I do this with multiple configurations where I set the lower dimension size L to different values.

Say I lower the dimensions of the points p and q to $\mathbb{R}^2$ and then to $\mathbb{R}^3$ and receive $p^{(2)}, q^{(2)}$ and $p^{(3)}, q^{(3)}$. Now I calculate the euclidean distance of $p^{(d)},q^{(d)} \: \: \forall d \in \{2,3\}$ to get $d^{(2)}$ and $d^{(3)}$.

Are these distances comparable, or will $d^{(3)}$ generally be bigger than $d^{(2)}$?

Background:

I am using a supervised model to cluster data points. The goal is to have tight clusters. I would like to compare different models, but the problem is that they output the data points in different number of dimensions, like illustrated:

enter image description here

My metric to minimize is the sum of the distances between the points and the centroid of the cluster:

$$ \sum_{i}^{N}\sqrt{\sum_{j}^{D}(x_{ij} - centroid_j)^2} $$

D is the number of dimensions and N the number of data points.

So my question is, how do I know which model is doing a better job in clustering the points? I am pretty sure that I can't simply compare the metric mentioned above. How could I achieve this?

$\endgroup$
  • $\begingroup$ You can't. Depending on the projection to 2d, you can get different results if you try to plot 3d data set on a plane. And, obviously, sum of squares of distances will be large in 3d than 2d. $\endgroup$ – Vasya Apr 3 at 22:50
  • $\begingroup$ Thank you for the response. I already thought about that. How would you then approach this problem? $\endgroup$ – oezguensi Apr 3 at 22:58
  • $\begingroup$ What do you mean by 'they output the data points in different dimensions'? I would have thought that a clustering model would take the data points as input and output a list detailing which data points belong to which cluster. $\endgroup$ – Angela Richardson Apr 12 at 6:05
  • 1
    $\begingroup$ To decide "if two euclidean distances are comparable if calculated on vectors from different dimensions" depends on how the points in the domain are mapped to the points in the codomain. If that's just what your black-box algorithm does then I think the answer is "no, they are not comparable". If you edit the question to tell us just how that mapping works maybe we can help. (Clarify with an edit, not in a comment.) $\endgroup$ – Ethan Bolker Apr 12 at 21:49
  • 1
    $\begingroup$ That's not enough information for me to help, If you edit the question to provide that material perhaps someone else can. If you can, post one or two nontrivial but not huge examples showing input and output. $\endgroup$ – Ethan Bolker Apr 13 at 2:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.