Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

A more general soft constraint is the Tikhonov regularization constraint $$ \mathbf{w}^\text{T}\Gamma^\text{T}\Gamma\mathbf{w} \leq C $$ which can capture relationships among the $w_i$ (the matrix $\Gamma$ is the Tikhonov regularizer).

(a) What should $\Gamma$ be to obtain a constraint of the form $\sum_{q=0}^Q w_q^2 \leq C$?

I think this is just the identity matrix since $\sum_{q=0}^Q w_q^2 = \mathbf{w}^\text{T}\mathbf{w}$

(b) What should $\Gamma$ be to obtain a constraint of the form $\left(\sum_{q=0}^Q w_q\right)^2 \leq C$?

To me, this is saying $\mathbf{ww} \leq C$. How is it possible to get $\mathbf{ww} = \mathbf{w}^\text{T}\mathbf{w}$ just by multiplying by some $\Gamma^\text{T}\Gamma$? Where am I going wrong here?

share|improve this question
add comment

1 Answer

up vote 1 down vote accepted

(a) You are right, in order to obtain $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\sum_{q=0}^Q w_q^2$, you should use $\Gamma=I$, where $I$ is the identity matrix.

(b) Be careful with your dimensions, $\mathbf{w}\mathbf{w}$ is not defined. You are trying to multiply a $Q\times 1$ matrix by a $Q\times 1$ matrix, which is not possible. To obtain $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\left(\sum_{q=0}^Q w_q\right)^2$, you should use $\Gamma=(1~ 1~\ldots~1)$, i.e. a row of ones. This implies that $\mathbf{w}^T\Gamma^T=\sum_{q=0}^Q w_q$, and thus that $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\left(\sum_{q=0}^Q w_q\right)^2$.

share|improve this answer
    
Couldn't we more generally take $\Gamma$ to be any orthogonal matrix? Then $\Gamma^T\Gamma=I$. –  icurays1 Mar 22 '13 at 1:58
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.