3
$\begingroup$

Let's say I have a matrix of data that is fully populated except for one value. For example, Column 1 is Height, Column 2 is Weight, Column 3 is Years Weight Lifting, and Column 4 is Bench Press Weight. So I surveyed 20 people and got their height, weight, years lifting, and bench press weight. Now I have a 5'11 individual weighing 170 pounds who has lifted for 1 year, and would like to predict his/her bench press weight. You could look at this as the matrix having a missing value, or you could look at it as wanting to predict a dependent variable given a vector of independent variables. There are curve fitting approaches to this kind of problem, but I would like to know how to use the Singular Value Decomposition to answer this question, if that is possible.

I am aware of the Singular Value Decomposition as a means of predicting missing values, but virtually all the information I have found has been in relation to huge, highly sparse matrices, with respect to the Netflix Prize and related problems. I cannot figure out how to use SVD or a similar approach to predict a missing value from a small or medium sized, fully populated (except for one missing value) matrix.

A step-by-step approach for solving the example above using SVD would be very helpful to me. Thank you!

$\endgroup$
1
$\begingroup$

Take a look at this paper. The basic idea is to solve the following optimization problem:

$$ \min \|X\|_*\\ \text{such that } X_{ij}=M_{ij}\quad \forall (i,j)\in\Omega $$ Here $X$ is the unknown matrix, $M$ is the known matrix, and $\Omega$ is the set of indicies known (in your case, $\Omega$ is all but one index). The special norm $\|X\|_*$ is the Nuclear Norm, defined using singular values as

$$ \|X\|_*=\sum_{j=1}^k\sigma_j $$ Here $k$ is the minimum dimension of your matrix. This optimization problem is convex, and the linked paper discusses solution details. Hope this helps!

$\endgroup$
  • 1
    $\begingroup$ It looks like the term "matrix completion" was a missing link in my searches. Although this paper still appears to be geared toward the Netflix Prize type of problem, this is a great starting point. Thank you! $\endgroup$ – Ricky Vesel Jun 30 '13 at 23:15
  • $\begingroup$ This method will still work, in fact it should work particularly well since your matrix is nearly complete. $\endgroup$ – icurays1 Jun 30 '13 at 23:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.