# Optimization, Gradients, and Multivariate Data

I would like to learn gradient based optimization for multivariate data.

For example, assume the data I have is $X = (x_0, ..., x_n)$ where $x_i$ are some random variables and $f$ a function measuring (Pearson, if you like) correlation. Then, I would like to minimize the value of $f(X)$ i.e. make the variables $x_0, ..., x_n$ uncorrelated. How could this be achieved using gradient based methods?

After I have learnt this, the next thing is that I would like to implement the procedure in MATLAB. If you have any tips for that, I would like to hear those as well.

-
What are the optimization variables here? Surely you don't want to change the input data arbitrarily, or I could throw away the original values and give you back random values for all the $x_i$. – Rahul Jan 5 '13 at 17:26
The example I gave is analogous to principal component analysis (PCA), which I know from statistics (and know how to do based on an eigenvalue decomposition). However, I am here interested to solve the problem I provided through gradient based methods, which are the subject of my interest. – user55285 Jan 5 '13 at 17:31
Are some variables random and some deterministic? Are you minimizing an expectation of sorts? – copper.hat Jan 5 '13 at 17:51