Kendall Tau and Pearson product moment are two (among many) correlation measures that quantify how closely one variable "tracks" another.
If you get the concept of an outlier, data points that are far from most other points, then the question arises as to how to weigh the outliers.
Pearson correlation weighs the outliers quadratically: outliers are relatively important.
Kendall Tau is called a "robust" correlation because it's based on just looking at rank-order as opposed to metric information and weighs outliers far less.
What's the difference? Think of a bicycle race. If only the order in which racers finish matters, rather than the clock-times (ie, metric) then it doesn't matter how far apart the racers are spaced - hence the winner could win by 1 second an hour (ie, outlier) - but the rank orders will stay the same.
The above example is just for the sake of understanding - there is no actual correlation with anything (you'd have to introduce another variable, maybe the degree of doping?)
There are many different ways to correlate data (some, like mutual information, quite abstract), each having its own limitations but how they weigh the data is a key distinction among them.
Let me add since you mention NLP, that one measure of document distance is via vector space models where correlations between words reflect how close documents match, see for example Rehurek "Scalability of semantic analysis...", 2011.