Let $x_i$ be a series of vectors of equal length, and let $\beta$ be a constant vector of equal length to $x_i$'s

I have the following sum

$$\sum_{i=1}^p (x_i^T \beta)^2 = \sum_{i=1}^p x_i^T \beta \beta^T x_i = x_1^T \beta \beta^T x_1 + x_2^T \beta \beta^T x_2 + \dots + x_p^T \beta \beta^T x_p$$

In order to apply a statistical theorem, I need to factorise this into a form

$$\sum_{i=1}^d T_i (x) g_i (\beta)$$

where each $T_i (x): \mathbb{R}^n \to \mathbb{R}$ out puts a single scalar $x$. and where the lower the $d$ the better. ie, I want to find the simplist factorisation of the above sum such that the $x$ terms and $\beta$ terms are separated.

My attempt:

I tried writing out the matrix multiplication $x_i^T \beta$ as the sum $\sum_j x_{ij}\beta_j$ but this didn't get me anywhere since it leads me to

$$\sum_j \sum_j \beta_j \beta_k \sum_i x_{ij}x_{ik}$$

which gives a total of $d=p^2$ summands... which is terrible considering $\beta$ is only of length $p$.

Any help here finding a simpler factorisation is much appreciated, thank you.

  • $\begingroup$ You can put the $x_i$ in a long vector after each other and then $\beta$ in another matrix $I_p \otimes \beta$ where $\otimes$ is Kronecker product. I can try write answer later today if you want. $\endgroup$ – mathreadler Oct 15 '18 at 5:50
  • $\begingroup$ wouldn't $(x_{i}^{t}\beta)$ be the inner product and be a scalar? $\endgroup$ – Shogun Oct 15 '18 at 5:52
  • $\begingroup$ hey @mathreadler thanks for introducing me to a new operation! Unfortunately I'm not sure that would work. The $T_i (x):\mathbb{R}^p \to \mathbb{R}$ are statistics and so must output a scalar, while in your form (if I understand correctly), $T_1 (x)$ would be a single vector of each $x$ end to end. This problem is related to sufficiency and the neyman factorisation theorem. I've editedmy post to clarify this $\endgroup$ – Xiaomi Oct 15 '18 at 7:03
  • $\begingroup$ @Xiaomi yes it is only part of solving the problem. like a hint that maybe can help you finish. i dont have time to write it all now as I am on work. I can write more later. $\endgroup$ – mathreadler Oct 15 '18 at 7:06
  • $\begingroup$ Oh, I see what you mean now. Thanks, I'll give that a think! $\endgroup$ – Xiaomi Oct 15 '18 at 7:07

Let us give a small example for $$I_p \otimes \beta$$ will be if p=3 and $\beta = [1,2,3]$:


We see that if we stuff $[x_1,x_2,x_3]^T$ into column vector we can do

$$(I_p \otimes \beta)[x_1,x_2,x_3]^T$$ and then we will get the 3 scalar products you have sought in resulting product vector. The only thing that remains is to take the squared two-norm of this vector.

$$\|(I_p \otimes \beta)[x_1,x_2,x_3]^T\|_2^2$$

It is known that $\|a\|_2^2=a^Ta$ so we can calculate this with for example:

$$([x_1,x_2,x_3](I_p \otimes \beta)^T)((I_p \otimes \beta)[x_1,x_2,x_3]^T)$$

And we have $\mathcal{FINISHED}$ :)

  • $\begingroup$ that's pretty cool $\endgroup$ – Shogun Oct 15 '18 at 22:22
  • $\begingroup$ Very interesting! Thank you! $\endgroup$ – Xiaomi Oct 16 '18 at 7:25
  • $\begingroup$ Pff you didn't really think I finished, did you? It would be 16 times overkill at this rate. $\endgroup$ – mathreadler Oct 16 '18 at 8:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.