Given a pair of strings in vector form $(s_i,s_j)$, I can find cosine similarity of pairs as follows:

$cosine(s_i,s_j)=s_i.*s_j / (\|s_i\|\|s_j\|)$

Similarly, bilinear similarity is defined as:

$BilinearSimilarity(s_i,s_j) = s_i^TWs_j $

I want to know what property of strings does the matrix $W$ captures intuitively? Why Should I prefer Bilinear similarity compared to cosine similarity?


Lets say your vectors $s_i \in \mathbb{R}^2$. Imagine that the only first component of your vector is meaningful but the second component is noise. When you compare two vectors $s_i$ and $s_j$, you should be comparing only their first components. This can be captured through bilinear similarity using $W = \begin{bmatrix}1 & 0\\0&0\end{bmatrix}$, because $$s_i^T W s_j = s_{i1}W_{11}s_{j1} = s_{i1}s_{j1}.$$

Next, consider a case where similarity along the first co-ordinate should be weighted more than similarity along the second co-ordinate. This is captured using $W = \begin{bmatrix}2 & 0\\0&1\end{bmatrix}$.

If you ignore the normalization, cosine similarity is just bilinear similarity with $W = \begin{bmatrix}1 & 0\\0&1\end{bmatrix}$.

Bilinear similarity is a generalization of cosine similarity where not all features are treated equal.

  • $\begingroup$ I know bilinear similarity reduces to cosine similarity when $W$ is identity matrix. I wanted to know if the role of the matrix $W$ is more than just weighing different features? Another angle to look at it is bilinear map is just a projection over some space and then computing cosine in that map. What advantage does the projection bring here? $\endgroup$ – chandresh Sep 16 '18 at 5:21
  • $\begingroup$ Yes, you can eigendecompose $W = U \Lambda U^T$ and think of bilinear similarity as a projection. The bilinear similarity is a more general inner product. The co-ordinates of $s$ may not be the correct space for an application, and you want to allow a transformed space. Imagine you have items with features $s_i$, and you've collected data by asking people "how similar is $i$ to $j$". You can fit a $W$ using this data and use it to estimate the similarity of two new items. $\endgroup$ – elexhobby Sep 16 '18 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.