Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

A more general soft constraint is the Tikhonov regularization constraint $$ \mathbf{w}^\text{T}\Gamma^\text{T}\Gamma\mathbf{w} \leq C $$ which can capture relationships among the $w_i$ (the matrix $\Gamma$ is the Tikhonov regularizer).

(a) What should $\Gamma$ be to obtain a constraint of the form $\sum_{q=0}^Q w_q^2 \leq C$?

I think this is just the identity matrix since $\sum_{q=0}^Q w_q^2 = \mathbf{w}^\text{T}\mathbf{w}$

(b) What should $\Gamma$ be to obtain a constraint of the form $\left(\sum_{q=0}^Q w_q\right)^2 \leq C$?

To me, this is saying $\mathbf{ww} \leq C$. How is it possible to get $\mathbf{ww} = \mathbf{w}^\text{T}\mathbf{w}$ just by multiplying by some $\Gamma^\text{T}\Gamma$? Where am I going wrong here?

share|cite|improve this question
up vote 1 down vote accepted

(a) You are right, in order to obtain $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\sum_{q=0}^Q w_q^2$, you should use $\Gamma=I$, where $I$ is the identity matrix.

(b) Be careful with your dimensions, $\mathbf{w}\mathbf{w}$ is not defined. You are trying to multiply a $Q\times 1$ matrix by a $Q\times 1$ matrix, which is not possible. To obtain $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\left(\sum_{q=0}^Q w_q\right)^2$, you should use $\Gamma=(1~ 1~\ldots~1)$, i.e. a row of ones. This implies that $\mathbf{w}^T\Gamma^T=\sum_{q=0}^Q w_q$, and thus that $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\left(\sum_{q=0}^Q w_q\right)^2$.

share|cite|improve this answer
Couldn't we more generally take $\Gamma$ to be any orthogonal matrix? Then $\Gamma^T\Gamma=I$. – icurays1 Mar 22 '13 at 1:58

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.