Note: I will be using capital letters to denote matrices, lowercase letters to denote vectors, and the Greek alphabet to denote any scalar quantities.

Note 2: I have tried my best to find a topic that deals with the same question I have, but I feel like there is not one. If you do know of one, please let me know.

I have been working on a project that, completely stripped down, revolves around a classic problem, which is ill-posed in my case:

$$A^HAx = A^Hb$$

where I use the superscript $^H$ to denote the (Hermitian) adjoint operator, i.e. the one that obeys $<Ax,y> \; = \; <x,A^Hy>$ for any x and y.

The vectors $x$ and $b$ above actually consist of smaller subvectors in concatenation:

$$A^HA[x_{1}^{T}, x_{2}^{T}, .., x_{n}^{T}]^{T} = A^{H}[b_{1}^{T},b_{2}^{T},..,b_{n}^{T}]^{T}$$

Up until now, I have been tackling the above problem with the conjugate gradients method, which has worked fine. I have obtained results that make some sense, but it definitely could do with further improvement. In particular, I have noticed that the outcome is less sparse than I would like. After some additional research, I have found that I can accurately predict the following quantity associated with the all the sub-vectors $x_{i}$:

$$R=\sum_{j=1}^J x_{j}^{2}\;/\; \sum_{j=1}^J |x_{j}|$$

(J being the size of one subvector), which is some (not very sophisticated) sparsity measure.

I am now wondering how I can incorporate this constraint in a regularization parameter to help guide the problem toward a better solution. I tried incorporating the above in Tikhonov regularization, instead trying to minimize:

$$||Ax-b||_{2}^{2} + \alpha||G x||_{2}^{2}$$

where G looks as follows

$$ \begin{bmatrix} |R_{ref,1}-R_{calc,1}|\,I & 0 & 0 \\ 0 & |R_{ref,2}-R_{calc,2}|\,I & 0 \\ 0 & 0 & |R_{ref,n}-R_{calc,n}|\,I \\ \end{bmatrix} $$

but, looking at the results for different $\alpha$'s, I feel like this still simply penalizes high-norm solutions, rather than deviations in the ratio I have defined.

My question therefore is: would anyone know of a way (or a reference/link) to better incorporate this specific knowledge in a regularization parameter, to help guide the solver? Thanks in advance!


What would be already tremendously helpful, is if I could punish deviations of the norm of the solution $||x||^2_{2}$ from a certain value. If such a thing exists, I will be happy to hear your suggestions.



You must log in to answer this question.