Tell me more ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.
up vote 2 down vote favorite

I would like to learn gradient based optimization for multivariate data.

For example, assume the data I have is $X = (x_0, ..., x_n)$ where $x_i$ are some random variables and $f$ a function measuring (Pearson, if you like) correlation. Then, I would like to minimize the value of $f(X)$ i.e. make the variables $x_0, ..., x_n$ uncorrelated. How could this be achieved using gradient based methods?

After I have learnt this, the next thing is that I would like to implement the procedure in MATLAB. If you have any tips for that, I would like to hear those as well.

share|improve this question
What are the optimization variables here? Surely you don't want to change the input data arbitrarily, or I could throw away the original values and give you back random values for all the $x_i$. – Rahul Narain Jan 5 at 17:26
The example I gave is analogous to principal component analysis (PCA), which I know from statistics (and know how to do based on an eigenvalue decomposition). However, I am here interested to solve the problem I provided through gradient based methods, which are the subject of my interest. – user55285 Jan 5 at 17:31
Are some variables random and some deterministic? Are you minimizing an expectation of sorts? – copper.hat Jan 5 at 17:51

1 Answer

up vote 0 down vote

You can use Gradient Descent method for optimization. It is one of the most common methods for optimization and learning and you can learn about it from various online resources. Further since you want to code this in matlab, their are two methods:

  • You start from point zero and write code based on mathematical algorithms you get online
  • Or you can refer to already written MATLAB code on gradient descent (http://www.mathworks.in/matlabcentral/fileexchange/38588-gradient-descent). And then extend on this code.

I have have good experience on such problems, so if you don't find anything productive online then please revert back as I can give a detailed algorithm with explanation to you.

share|improve this answer
I have already searched for resources online. After reading the material I could find, I did not succeed to implement the example I gave. That's why I am here, I would like to learn how the gradient based optimization methods would work for multivariate data starting from the beginning and assuming the (simple?) example I gave. – user55285 Jan 5 at 17:26
Fine! I will get back to you with a detailed document/explanation in some time until then you can look on the gradient descent code, whose link I gave to you. As I have seen this code is quite self explanatory! – Akshay Jan 5 at 17:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.