# Linear algebra: finding a Tikhonov regularizer matrix

A more general soft constraint is the Tikhonov regularization constraint $$\mathbf{w}^\text{T}\Gamma^\text{T}\Gamma\mathbf{w} \leq C$$ which can capture relationships among the $w_i$ (the matrix $\Gamma$ is the Tikhonov regularizer).

(a) What should $\Gamma$ be to obtain a constraint of the form $\sum_{q=0}^Q w_q^2 \leq C$?

I think this is just the identity matrix since $\sum_{q=0}^Q w_q^2 = \mathbf{w}^\text{T}\mathbf{w}$

(b) What should $\Gamma$ be to obtain a constraint of the form $\left(\sum_{q=0}^Q w_q\right)^2 \leq C$?

To me, this is saying $\mathbf{ww} \leq C$. How is it possible to get $\mathbf{ww} = \mathbf{w}^\text{T}\mathbf{w}$ just by multiplying by some $\Gamma^\text{T}\Gamma$? Where am I going wrong here?

-

(a) You are right, in order to obtain $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\sum_{q=0}^Q w_q^2$, you should use $\Gamma=I$, where $I$ is the identity matrix.
(b) Be careful with your dimensions, $\mathbf{w}\mathbf{w}$ is not defined. You are trying to multiply a $Q\times 1$ matrix by a $Q\times 1$ matrix, which is not possible. To obtain $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\left(\sum_{q=0}^Q w_q\right)^2$, you should use $\Gamma=(1~ 1~\ldots~1)$, i.e. a row of ones. This implies that $\mathbf{w}^T\Gamma^T=\sum_{q=0}^Q w_q$, and thus that $\mathbf{w}^T\Gamma^T \Gamma \mathbf{w}=\left(\sum_{q=0}^Q w_q\right)^2$.
Couldn't we more generally take $\Gamma$ to be any orthogonal matrix? Then $\Gamma^T\Gamma=I$. – icurays1 Mar 22 '13 at 1:58