# How to 'minimize' correlation between series

Hi fellow mathemagicians,

let's say that I have 3 series of numerical results (they represent 'drawdowns') :

S1   S2   S3
0    0    0
0   -5    0
0  -10  -10
-5    0  -15
0   -5   -5


We can determine their correlation matrix :

          S1          S2          S3
S1            1  -0.534522484 0.771743633
S2 -0.534522484             1 0.045834925
S3  0.771743633   0.045834925           1


and then I would need some function that will "minimize the overall correlation" depending on unknown yet weightings to apply to each series.

In other words, I would like to find the best combination of weightings to apply to each series so that the sum of the 3 adjusted values for each line is 'the lowest possible over all the results'.

I would like to use an optimizer that will "minimize their overall correlation" and for that, I need an objective function which I just described. There would also be a constraint on the sum of the weightings to be equal to 1 and another constraint for achieving a minimum profit level.

So I would need help writing the objective function (in AMPL ideally) of my problem, that will minimize the overall correlation between my drawdowns series :)

Although I am am still interested in using an optimizer,

Following T.K's answer, I decided to try removing the correlation with the linear transformation he kindly pointed at on wikipedia's page :

I have my matrix X and I am able to find the matrix D. However, I am encountering a problem with the matrix square root operator...I don't know how to compute it :\

Here is what I got so far :

In order to get $T$, I need to apply the matrix square root operator on the matrix presented at the bottom right of the image, which is the inverse matrix of $D^TD$.

-

For the last (and important) edit/answer see the text after the last horizontal line

I'm pretty sure, the formula for the correlation as given by Wikipedia helps you a lot: https://secure.wikimedia.org/wikipedia/en/wiki/Correlation

Especially the sample correlation coefficient. As well it might be helpful to use the linearity of the expected value: $$\mathrm{E}[\alpha X]=\alpha \mathrm{E}[X]$$ and the variance: $$\mathrm{Var}(\alpha X)=\alpha^2 \mathrm{Var}(X)$$

From these three formulas (sample correlation coefficient, expected value and variance) you should be able to derive your objective function.

Some more details (formulas mainly taken from above link):

The correlation of random variables ($corr(X,Y)$) is their covariance ($\mathrm{Cov}(X,Y)$) divided by the product of their standard deviation ($\sigma_X\sigma_Y$), which is the product of the square roots of their variances ($\mathrm{Var}(X)=\sigma_X^2$, $\mathrm{Var}(Y)=\sigma_Y^2$), thus $$\mathrm{corr}(X,Y)=\frac{\mathrm{Cov}(X,Y)}{\sqrt{\mathrm{Var}(X)}\sqrt{\mathrm{Var}(Y)}}$$

The covariance of two random variables can be rewritten in terms of the expected value: $$\mathrm{Cov}(X,Y)=\mathrm{E}[(X-\mathrm{E}[X])(Y-\mathrm{E}[Y])]=\mathrm{E}[XY]-\mathrm{E}[X]\cdot \mathrm{E}[Y]$$

Now one can introduce some weights $\alpha$ and $\beta$ to the random variables.
As the covariance is derived from the expected value, one gets the linearity $$\mathrm{Cov}(\alpha X,\beta Y)=\alpha\beta \mathrm{Cov}(X,Y)$$

With this, one gets for the correlation: $$\mathrm{corr}(\alpha X,\beta Y)=\frac{\alpha\beta \mathrm{Cov}(X,Y)}{\sqrt{\alpha^2\mathrm{Var}(X)}\sqrt{\beta^2\mathrm{Var}(Y)}}$$

Finally, you insert your random variables $S_1$, $S_2$ and $S_3$ with their weights $\alpha$, $\beta$ and $\gamma$ into this last formula and come up with three equations.

While reading some details on correlation matrices, I just discovered a way solving your problem, without using a minimiser:
Transformation of random variables (see https://secure.wikimedia.org/wikipedia/en/wiki/Pearson_product-moment_correlation_coefficient#Removing_correlation)

Applying your example to the description given by Wikipedia, $n$ is 3 ($S_1$ to $S_3$) and $m$ is 5. $X$ is the matrix you gave here at the very top.

On your problem with the matrix square root.
I've done all calculations in R as I'm not familiar with doing it in Excel. Maybe somebody else can help you out with that.

> X
[,1] [,2] [,3]
[1,]    0    0    0
[2,]    0   -5    0
[3,]    0  -10  -10
[4,]   -5    0  -15
[5,]    0   -5   -5
> Z
[,1] [,2] [,3] [,4] [,5]
[1,]    1    1    1    1    1
[2,]    1    1    1    1    1
[3,]    1    1    1    1    1
[4,]    1    1    1    1    1
[5,]    1    1    1    1    1
> D = X - (1/5)*Z %*% X
> D
[,1] [,2] [,3]
[1,]    1    4    6
[2,]    1   -1    6
[3,]    1   -6   -4
[4,]   -4    4   -9
[5,]    1   -1    1
> DTD = solve(t(D) %*% D)
> DTD.eig = eigen(DTD)
> DTD.sqrt = DTD.eig$vectors %*% diag(sqrt(DTD.eig$values)) %*% solve(DTD.eig$vectors) > DTD.sqrt [,1] [,2] [,3] [1,] 0.7479411 0.19820775 -0.18609441 [2,] 0.1982077 0.17628059 -0.05451766 [3,] -0.1860944 -0.05451766 0.12541941 > T = D %*% DTD.sqrt > T [,1] [,2] [,3] [1,] 0.4242056 0.57622412 0.348351376 [2,] -0.5668331 -0.30517882 0.620939695 [3,] 0.3030723 -0.64140513 -0.360666057 [4,] -0.5240837 0.40295033 -0.602467673 [5,] 0.3636389 -0.03259050 -0.006157341  Check the result: The covariance matrix of$T$should be the identity matrix. > cov(T) [,1] [,2] [,3] [1,] 2.500000e-01 -7.769177e-17 -1.301994e-16 [2,] -7.769177e-17 2.500000e-01 -6.960356e-16 [3,] -1.301994e-16 -6.960356e-16 2.500000e-01  The off-diagonal elements are all almost zero (rounding error of floating-point arithmetic). The diagonal elements are$0.25$, so there must be still some error in my calculation. I'm sorry for that. Maybe somebody else finds that error? Just another edit after having a chat with a colleague about this topic: First a little remark I was not aware of yet: It is sufficient to look at the covariance-variance matrix instead of the correlation matrix, as the correlation of two variables can only be$0$if and only if the covariance of those two variables is$0$as well. (as the formula for the correlation is derived from the formula for covariance, see above) Now I'm pretty sure, that you definitely do not need an optimiser and that you will not find one single set$(\alpha,\beta,\gamma)$to minimise the correlation of the three (or more) series. What you want is applying a Principal Component Analysis (PCA) on your data. You will get (at most) three valid sets$(\alpha,\beta,\gamma)$but should only take the one, which describes the highest proportion of variance of your data. The coefficients of the first principal component are your set$(\alpha,\beta,\gamma)$. If you do not have a built-in function in your program for calculating a PCA, you do the following: 1. write your data as a matrix$S$with each variable (series) as one column 2. calculate the covariance-variance matrix$\Sigma$of your data by $$\Sigma=S^TS$$ 3. calculate the eigenvalues and eigenvectors of$C$(various ways of doing this) 4. the eigenvector corresponding to the highest eigenvalue is the set of$(\alpha,\beta,\gamma)$, which describes most of the variance of the original data Now you have (three) linear combinations of your original variables (series), where the covariance (and consequently the correlation) between the different combinations is zero (by definition and methodology). As I do not know what exact application your initial problem has, I want to refer to the last paragraph of the Details-section of the linked Wikipedia article. - Thank you very much for your reply. I believe I will need more direct instructions though as I am not a expert in maths myself. Isn't the correlation formula good for getting a correlation value between 2 series only? How to calculate it, or 'its equivalent', for 3 or more? Could you also explain more thoroughly please how I can make use of the 3 formulas you kindly provided in order to derive my objective function? Thanks again for your time :) – ibiza Apr 27 '11 at 12:32 @ibiza: I expanded my post. Have a look on the last (3rd) section. It will probably solve your problem without utilising an optimiser. I'm just not quite sure, whether the diagonal elements of the matrix$T$will be your weights. Though the weights should be somewhere in$T$(just my intuition). – Torbjörn Apr 27 '11 at 14:52 @T.K. Hi and thanks for your thoughtful post. I tried to use the linear transformation you suggested but am encountering one problem, could you help me compute the last part that I am struggling with? – ibiza Apr 27 '11 at 16:40 @T.K. Also, in the second section of your answer, about minimiser, you end it with "and come up with three equations". I apologize for my lack of understanding, but I would need to know what to do with these 3 equations? Because in the end, I want my objective function to be a single equation to minimize, how would that be possible? Thanks again for your time, it is deeply appreciated. – ibiza Apr 27 '11 at 16:54 @ibiza: See the R output. That's your solution. (as it would be done in R) About the three equations: They can be turned into one single equation but in matrix-vector notation – Torbjörn Apr 27 '11 at 17:28 This should go as a comment but seems to be too long for that field I've got the matrix-squareroot$ M=\sqrt{(D^t D)^{-1}}$numerically to be approximately $$\begin{bmatrix} 0.747941101687 & 0.198207748957 & -0.186094411506 \\ 0.198207748957 & 0.176280588608 & -0.0545176638058 \\ -0.186094411506 & -0.0545176638058 & 0.125419407082 \end{bmatrix}$$ using Pari/GP and diagonalization - Thanks for that, @Gottfried. I do not see any big differences between our results, do you? – Torbjörn Apr 27 '11 at 18:26 @T.K.:arrggh... For whatever reason - I did not see that matrix in your comment before, sorry, must have been blind... – Gottfried Helms Apr 27 '11 at 18:30 @T.K.: Second view: just found in the revision-story of your msg that your correction and my msgs crossed at the same time ... – Gottfried Helms Apr 27 '11 at 18:37 @Gottfried: No worries :) – Torbjörn Apr 27 '11 at 18:39 @Gottfried Thanks for your contribution :) It is still puzzling however why the covariance matrix of$T\$ is not equal to the identity matrix, as stated that it should be in the wikipedia article and method found by T.K... – ibiza Apr 27 '11 at 18:42