How to increase the correlation?

I have three vectors of numbers with the same dimensionality, $A$,$B$ and $C$. What is the most suitable number $x$, which maximizes the correlation of $A$ and $B+xC$ . To what extend can I increase the correlation.

Thanks

• Does correlation means inner product? Or are $A, B$ and $C$ random vectors, that is, $n$-tuples whose components are random variables? – Dilip Sarwate Dec 2 '12 at 12:27

(I interpret correlation as described here.) Since the correlation does not change if you displace any of these vectors by a multiple of the constant vector $\mathbf{1}$ we can assume that $$\langle A, \mathbf{1} \rangle = \langle B, \mathbf{1} \rangle = \langle C, \mathbf{1} \rangle = 0.$$ (That is, we can assume that each of these vectors has zero mean.) Then the correlation between $A$ and $B+xC$ is

$$\operatorname{Corr}(A, B+xC) = \frac{\langle A, B + xC \rangle}{||A|| \cdot ||B + x C||}$$

The derivative in $x$ of this expression is

$$\frac{\langle A,C \rangle \langle B, B \rangle - \langle A, B \rangle \langle B, C \rangle + x (\langle A, C \rangle \langle B, C \rangle - \langle A, B \rangle \langle C, C \rangle)}{||A|| \cdot ||B + x C||^3}$$

and therefore if the correlation is maximal or minimal then

$$x = \frac{\langle A,C \rangle \langle B, B \rangle - \langle A, B \rangle \langle B, C \rangle}{\langle A, B \rangle \langle C, C \rangle - \langle A, C \rangle \langle B, C \rangle} = \frac{||B||}{||C||} \cdot \frac{\operatorname{Corr}(A,C) - \operatorname{Corr}(A,B) \operatorname{Corr}(B,C)}{\operatorname{Corr}(A,B) - \operatorname{Corr}(A,C) \operatorname{Corr}(B,C)}.$$

If the correlation has a maximum then the correlation as function in $x$ never reaches a minimum and vice versa. Both situations can occur (e.g. replace $A$ by $-A$).

Edit: (Exposition of the derivative.) Substitute $x \leftarrow x+t$ in the expression for the correlation to obtain:

$$\frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot \left(||B + x C||^2 + 2t \langle B + xC, C \rangle + O(t^2)\right)^{1/2}} =$$

$$\frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot ||B + x C|| \cdot \left(1 + t \frac{\langle B + xC, C \rangle}{||B+xC||^2} + O(t^2)\right)} =$$

$$\frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot ||B + x C||} \left(1 - t \frac{\langle B + xC, C \rangle}{||B+xC||^2} + O(t^2)\right) =$$

$$\frac{\langle A, B + xC \rangle}{||A|| \cdot ||B + x C||} + t \left( \frac{\langle A, C \rangle}{||A|| \cdot ||B + x C||} - \frac{\langle A, B + xC \rangle \langle B + xC, C \rangle}{||A|| \cdot ||B + x C||^3}\right) + O(t^2)$$

Therefore the derivative in $x$ of the correlation is

$$\frac{\langle A, C \rangle}{||A|| \cdot ||B + x C||} - \frac{\langle A, B + xC \rangle \langle B + xC, C \rangle}{||A|| \cdot ||B + x C||^3}$$

which after some rewriting is the expression above. Note that in particular the $x^2$ term in the numerator cancels.

• Would you please describe how did you derive? – remo Dec 2 '12 at 16:25
• @remo I added a section to compute the derivative. – WimC Dec 2 '12 at 17:29
• Thanks. I wonder I you calculated them using pen and paper or a software. – remo Dec 2 '12 at 19:35
• @remo Pen and paper. (Fingers crossed.) – WimC Dec 2 '12 at 19:35
• I had a hard time to find a solution with pen and paper and first I could not believe that the solution is that nice. I tested it with a script and it seems to be correct. That substitution with x+t is an awesome trick – idclev 463035818 May 28 '16 at 6:13