# Minimization of function expressed with vectors and matrices

I need to find vector $\bf{p}$ in the following system:

$$\bf{0} \approx \bf{W} \left[ \bf{C}^2 \bf{p} - \bf{d} \right]$$

$$\bf{0} \approx \varepsilon \bf{p}$$

In the above, $\bf{0}$ is a vector, $\bf{W}$ is a matrix, $\bf{C}$ is a matrix, $\bf{d}$ is a vector, $\epsilon$ is a scalar, and all of these are known inputs.

Perhaps the first and easiest way to proceed would be to re-write the system as the following:

$$\bf{0} \approx {\bf{W}}\left[ \bf{C}^2 \bf{p} - \bf{d} \right] + \varepsilon \bf{p}$$

How would I proceed to be able to find a $\bf{p}$ vector that will minimize the system? I am prototyping this calculation in Matlab, but I need to re-write the calculation in C++. Could anyone suggest an optimization algorithm (with some sort of numerical library) that I can use to minimize? I believe that the optimization algorithms bundled with Matlab require a scalar output of a function to be minimized.

As a reference, this system of equations is given in the following paper: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.8057&rep=rep1&type=pdf

-
It is not clear what you are trying to do. What do you mean by the $\approx$ symbol? –  copper.hat May 11 '12 at 16:52
@copper.hat: I want to find a vector $\bf{p}$ that will minimize the RHS of the expression. The notation follows this paper: citeseerx.ist.psu.edu/viewdoc/…. –  Nicholas Kinar May 11 '12 at 20:55
The paper doesn't define what it means by $\approx$ other than 'minimizing the residual'. There are many ways of doing this. If you choose the $2$-norm, then you could use least squares. I can't imagine what $0 \approx \epsilon p$ means, other than some regularization, or attempt to limit $p$ in some manner. –  copper.hat May 11 '12 at 21:17
@copper.hat: Sure, that sounds reasonable; thanks for suggesting this. How would I set up the math so that I can use least squares to find a minimum $\bf{p}$? Maybe adding the two equations together as I did in my question above is the way to get rid of the $\bf{0} \approx \varepsilon \bf{p}$? I don't know how to limit $\bf{p}$, but I have to say that much is left to our imaginations. –  Nicholas Kinar May 12 '12 at 0:51
You could minimize $||W(C^2p-d)||_2^2+||\epsilon p||_2^2$. Look for least squares routines, or just solve the normal equations (not necessarily the best way, but easy to set up). –  copper.hat May 12 '12 at 1:00

Try solving the least-squares problem:

$$\min_{p} \; \; ||\begin{bmatrix} W C^2 \\ \epsilon I \end{bmatrix}p - \begin{bmatrix} W d \\ 0 \end{bmatrix} ||_2^2$$

-
Thank you; do I have to use the squared norm to be able to find a solution using an algorithm such as the one given on pg. 5 of the least squares tutorial (maths.lth.se/matematiklth/personal/fredrik/kombopt/f10eng.pdf)? I believe that the solution given on pg. 5 of the tutorial does not square the norm. –  Nicholas Kinar May 12 '12 at 19:38
Since the square root is strictly increasing (for $x\geq 0$), minimizing either will have the same effect. As an aside, the result on pg. 5 was derived using the norm squared. –  copper.hat May 12 '12 at 19:52
Thanks again for all of your help. I think that I now understand what is going on here. –  Nicholas Kinar May 12 '12 at 20:01