I have three vectors of numbers with the same dimensionality, $A$,$B$ and $C$. What is the most suitable number $x$, which maximizes the correlation of $A$ and $B+xC$ . To what extend can I increase the correlation.


  • $\begingroup$ Does correlation means inner product? Or are $A, B$ and $C$ random vectors, that is, $n$-tuples whose components are random variables? $\endgroup$ – Dilip Sarwate Dec 2 '12 at 12:27

(I interpret correlation as described here.) Since the correlation does not change if you displace any of these vectors by a multiple of the constant vector $\mathbf{1}$ we can assume that $$\langle A, \mathbf{1} \rangle = \langle B, \mathbf{1} \rangle = \langle C, \mathbf{1} \rangle = 0.$$ (That is, we can assume that each of these vectors has zero mean.) Then the correlation between $A$ and $B+xC$ is

$$ \operatorname{Corr}(A, B+xC) = \frac{\langle A, B + xC \rangle}{||A|| \cdot ||B + x C||} $$

The derivative in $x$ of this expression is

$$ \frac{\langle A,C \rangle \langle B, B \rangle - \langle A, B \rangle \langle B, C \rangle + x (\langle A, C \rangle \langle B, C \rangle - \langle A, B \rangle \langle C, C \rangle)}{||A|| \cdot ||B + x C||^3} $$

and therefore if the correlation is maximal or minimal then

$$ x = \frac{\langle A,C \rangle \langle B, B \rangle - \langle A, B \rangle \langle B, C \rangle}{\langle A, B \rangle \langle C, C \rangle - \langle A, C \rangle \langle B, C \rangle} = \frac{||B||}{||C||} \cdot \frac{\operatorname{Corr}(A,C) - \operatorname{Corr}(A,B) \operatorname{Corr}(B,C)}{\operatorname{Corr}(A,B) - \operatorname{Corr}(A,C) \operatorname{Corr}(B,C)}. $$

If the correlation has a maximum then the correlation as function in $x$ never reaches a minimum and vice versa. Both situations can occur (e.g. replace $A$ by $-A$).

Edit: (Exposition of the derivative.) Substitute $x \leftarrow x+t$ in the expression for the correlation to obtain:

$$ \frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot \left(||B + x C||^2 + 2t \langle B + xC, C \rangle + O(t^2)\right)^{1/2}} = $$

$$ \frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot ||B + x C|| \cdot \left(1 + t \frac{\langle B + xC, C \rangle}{||B+xC||^2} + O(t^2)\right)} = $$

$$ \frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot ||B + x C||} \left(1 - t \frac{\langle B + xC, C \rangle}{||B+xC||^2} + O(t^2)\right) = $$

$$ \frac{\langle A, B + xC \rangle}{||A|| \cdot ||B + x C||} + t \left( \frac{\langle A, C \rangle}{||A|| \cdot ||B + x C||} - \frac{\langle A, B + xC \rangle \langle B + xC, C \rangle}{||A|| \cdot ||B + x C||^3}\right) + O(t^2) $$

Therefore the derivative in $x$ of the correlation is

$$ \frac{\langle A, C \rangle}{||A|| \cdot ||B + x C||} - \frac{\langle A, B + xC \rangle \langle B + xC, C \rangle}{||A|| \cdot ||B + x C||^3} $$

which after some rewriting is the expression above. Note that in particular the $x^2$ term in the numerator cancels.

  • $\begingroup$ Would you please describe how did you derive? $\endgroup$ – remo Dec 2 '12 at 16:25
  • $\begingroup$ @remo I added a section to compute the derivative. $\endgroup$ – WimC Dec 2 '12 at 17:29
  • $\begingroup$ Thanks. I wonder I you calculated them using pen and paper or a software. $\endgroup$ – remo Dec 2 '12 at 19:35
  • $\begingroup$ @remo Pen and paper. (Fingers crossed.) $\endgroup$ – WimC Dec 2 '12 at 19:35
  • $\begingroup$ I had a hard time to find a solution with pen and paper and first I could not believe that the solution is that nice. I tested it with a script and it seems to be correct. That substitution with x+t is an awesome trick $\endgroup$ – idclev 463035818 May 28 '16 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.