Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have three vectors of numbers with the same dimensionality, $A$,$B$ and $C$. What is the most suitable number $x$, which maximizes the correlation of $A$ and $B+xC$ . To what extend can I increase the correlation.


share|cite|improve this question
Does correlation means inner product? Or are $A, B$ and $C$ random vectors, that is, $n$-tuples whose components are random variables? – Dilip Sarwate Dec 2 '12 at 12:27
up vote 2 down vote accepted

(I interpret correlation as described here.) Since the correlation does not change if you displace any of these vectors by a multiple of the constant vector $\mathbf{1}$ we can assume that $$\langle A, \mathbf{1} \rangle = \langle B, \mathbf{1} \rangle = \langle C, \mathbf{1} \rangle = 0.$$ (That is, we can assume that each of these vectors has zero mean.) Then the correlation between $A$ and $B+xC$ is

$$ \operatorname{Corr}(A, B+xC) = \frac{\langle A, B + xC \rangle}{||A|| \cdot ||B + x C||} $$

The derivative in $x$ of this expression is

$$ \frac{\langle A,C \rangle \langle B, B \rangle - \langle A, B \rangle \langle B, C \rangle + x (\langle A, C \rangle \langle B, C \rangle - \langle A, B \rangle \langle C, C \rangle)}{||A|| \cdot ||B + x C||^3} $$

and therefore if the correlation is maximal or minimal then

$$ x = \frac{\langle A,C \rangle \langle B, B \rangle - \langle A, B \rangle \langle B, C \rangle}{\langle A, B \rangle \langle C, C \rangle - \langle A, C \rangle \langle B, C \rangle} = \frac{||B||}{||C||} \cdot \frac{\operatorname{Corr}(A,C) - \operatorname{Corr}(A,B) \operatorname{Corr}(B,C)}{\operatorname{Corr}(A,B) - \operatorname{Corr}(A,C) \operatorname{Corr}(B,C)}. $$

If the correlation has a maximum then the correlation as function in $x$ never reaches a minimum and vice versa. Both situations can occur (e.g. replace $A$ by $-A$).

Edit: (Exposition of the derivative.) Substitute $x \leftarrow x+t$ in the expression for the correlation to obtain:

$$ \frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot \left(||B + x C||^2 + 2t \langle B + xC, C \rangle + O(t^2)\right)^{1/2}} = $$

$$ \frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot ||B + x C|| \cdot \left(1 + t \frac{\langle B + xC, C \rangle}{||B+xC||^2} + O(t^2)\right)} = $$

$$ \frac{\langle A, B + xC \rangle + t \langle A, C \rangle}{||A|| \cdot ||B + x C||} \left(1 - t \frac{\langle B + xC, C \rangle}{||B+xC||^2} + O(t^2)\right) = $$

$$ \frac{\langle A, B + xC \rangle}{||A|| \cdot ||B + x C||} + t \left( \frac{\langle A, C \rangle}{||A|| \cdot ||B + x C||} - \frac{\langle A, B + xC \rangle \langle B + xC, C \rangle}{||A|| \cdot ||B + x C||^3}\right) + O(t^2) $$

Therefore the derivative in $x$ of the correlation is

$$ \frac{\langle A, C \rangle}{||A|| \cdot ||B + x C||} - \frac{\langle A, B + xC \rangle \langle B + xC, C \rangle}{||A|| \cdot ||B + x C||^3} $$

which after some rewriting is the expression above. Note that in particular the $x^2$ term in the numerator cancels.

share|cite|improve this answer
Would you please describe how did you derive? – remo Dec 2 '12 at 16:25
@remo I added a section to compute the derivative. – WimC Dec 2 '12 at 17:29
Thanks. I wonder I you calculated them using pen and paper or a software. – remo Dec 2 '12 at 19:35
@remo Pen and paper. (Fingers crossed.) – WimC Dec 2 '12 at 19:35
I had a hard time to find a solution with pen and paper and first I could not believe that the solution is that nice. I tested it with a script and it seems to be correct. That substitution with x+t is an awesome trick – tobi303 May 28 at 6:13

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.