The application of NMF is related to extracting features in original data present as a matrix. An important problem when utilizing NMF for feature extraction is the choice of rank r.

However I can't understand what the features in the original matrix actually are - are they related to the values of each matrix element, or are the features common to columns/rows? Or are the features patterns across the entire 2D matrix?

The reason I'm asking is that it feels like utilizing NMF is essentially impossible without this, since constructing the input matrix as well as deciding the rank r becomes impossible.


NMF became especially popular after the Netflix contest, however, it has been around for quite a while. A good paper to get started with NMF would be read Algorithms for Non-negative Matrix Factorization by Lee and Seung.

I will try to explain NMF in the context of a movie recommendation system. So imagine you are on Netflix and you can either like a movie or dislike a movie. Let's say that corresponds to a rating of $1$ or $0$ respectively (this is an example of binary NMF which has its own suite of algorithms that I encourage you to read). So what you can do is construct a matrix $A$ whose rows would correspond to each user and the columns would have all the observable movies on Netflix. Thus $A_{ij}$ corresponds to the rating given by user $i$ about movie indexed $j.$ Usually, these matrices have huge dimensions and as you can quickly observe, they are very sparse since you have not seen all the movies, let alone rate them.

Now what you want to do is some form of dimensionality reduction and here is where you start getting different set of algorithms. On classical method is to basically factorize $A \approx P^{t} Q.$ I use the approximate symbol instead of equality since you will learn the matrices $P$ and $Q.$ How do you do this? Well, you simply define an error function. A simple one could be just minimizing the Frobenius norm and so $$\min_{P, Q}||A-P^tQ||.$$ With the error function defined you can gradient descent and hopefully converge to an approximation of $P$ and $Q.$

The number of columns of the matrix $P^{t}$ are called latent features, which in some sense represent the dimension of the reduced space. So what's the advantage of approximating $A$ by $P^{t}Q?$ Well, after performing NMF and computing the product $P^{t}Q$ you will get a matrix which is actually not sparse like $A$ and this is great since now you have learned the ratings of the user for movies that he has not yet seen. With this, you can simply rank the movies by ratings and give the appropriate recommendation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.