Given the dataset $\{(X_1, y_1), (X_2, y_2), ..., (X_n, y_n)\}$, where $X_i$ are matrices of identical size, and $y_i$ are scalers, propose the following two-way linear regression scheme: $$ \hat{y_i} = u^{T} X_i v, $$ where $u$ and $v$ are vectors of appropriate sizes. Errors are defined in least square, so $$ \min_{u,v} \sum_{i} \left(y_i - \hat{y_i}\right)^2. $$

  1. What is the general, linear algebra solution for such regression, if there is one?
  2. Suggestions on solving this numerically by computer with the extra constraints that $||u|| = 1$ and all entry in $v$ is nonnegative, given that all $y_i$ is nonnegative?

1 Answer 1


An interesting problem. What is motivation for such a regression? In any case, even at point 1. the problem is not linear. The minimized function is given by \begin{align} f:=\frac{1}{2}\sum_{i=1}^n\left(y_i-\sum_{k,l}u_kX_{i,kl}v_l\right)^2 \end{align} and necessary first order conditions have the form: \begin{align} \frac{\partial f}{\partial u_K}=\sum_{i=1}^n\left[\left(y_i-\sum_{k,l}u_kX_{i,kl}v_l\right)\sum_lX_{i,Kl}v_l\right]=0,\\ \frac{\partial f}{\partial v_L}=\sum_{i=1}^n\left[\left(y_i-\sum_{k,l}u_kX_{i,kl}v_l\right)\sum_lX_{i,kL}u_k\right]=0, \end{align} i.e. you get a set of quadratic equations for which you need a nonlinear solver. At point 2., in order to keep non-negativity of $u$ and $v$, you need even a nonlinear optimizer, which is able to handle inequalities.

  • $\begingroup$ Sorry for the late reply. The motivation comes from a biological question we are solving. Basically, there are two sets of individual effects (in our case, different cellular receptors and cell types), and their total effect can be represented as a product of two. This also reduces the number of unnecessary parameters to be fit (as the dataset is not that big in our case). I propose it to be a general question because I think there should be many cases where there are two sets of independent effects where the overall effect is just a product of two. $\endgroup$ Jan 18 at 20:42
  • $\begingroup$ Did my answer helped you in your problem? Have you tried to solve these equations? $\endgroup$ Jan 18 at 21:24
  • 1
    $\begingroup$ Yes, I'm still in process of solving these. I think there should be a solution involving matrix/tensor derivative which I'm working on right now, and I wish it can guide some useful constraint to tame this question. $\endgroup$ Jan 18 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.