# Multiple vector rejection from multiple vectors - possible with matrix notation?

Assume $\vec{x}$ and $\vec{y}$ are vectors. We can obtain the projection of vector $\vec{x}$ onto $\vec{y}$ by using the following formula:

$proj_{\vec{x}->\vec{y}} = \frac{\vec{x}\cdot \vec{y}}{\vec{y} \cdot \vec{y}}\vec{y}$

And the rejection of vector $\vec{x}$ onto $\vec{y}$ is

$rejection_{\vec{x}->\vec{y}} = \vec{x} - proj_{\vec{x}->\vec{y}} = \vec{x} - \frac{\vec{x}\cdot \vec{y}}{\vec{y} \cdot \vec{y}}\vec{y}$

Is there a matrix generalization of vector rejection? To say it in more detail:

Assume that $\vec{x}$ is a vector and that the matrix $Y$ represents multiple vectors $\vec{y}_i$. Is there an easy way to calculate the vector rejection of $\vec{x}$ from each vector in $Y$ such that the resulting $\vec{x}$ vector is orthogonal to all vectors $\vec{y}$ in $Y$?

Assume that the matrix $X$ represents multiple vectors $\vec{x}_i$ and that the matrix $Y$ represents multiple vectors $\vec{y}_i$. Is there an easy way to calculate the vector rejection of each vector $\vec{x}_i$ from each vector $\vec{y}_i$ such that each vector $\vec{x}_i$ in $X$ is orthogonal to each vector $\vec{y}_i$ in $Y$?

• A projection can be represented with a matrix $\bf P$, the rejection is then done by the matrix $({\bf I-P})$ where $\bf I$ is the unit matrix which leaves vectors alone when you multiply with it. – mathreadler May 23 '17 at 13:49
• @mathreadler Ok, so how do I find $P$? – Paul Terwilliger May 23 '17 at 13:59

If the matrix $P$ represents orthogonal projection onto $U=\operatorname{span}\{u_1,\dots,u_k\}$, then $P_\perp=I-P$ represents represents orthogonal rejection, which is the same as orthogonal projection onto $V^\perp$. We have $PP_\perp v=P(I-P)v=Pv-P^2v=Pv-Pv=0$, so $P_\perp v$ is orthogonal to $U$, and obviously $v=Pv+P_\perp v$.

There are several ways to construct an orthogonal projection matrix $P$, depending on what you’re starting with. If $u_1,\dots,u_k$ form an orthogonal basis for $U$, then you can simply add up the $k$ individual projection matrices. If they’re not orthogonal but are linearly independent, you can use the formula $P=U(U^TU)^{-1}U^T$, where (with a bit of abuse of notation) $U$ is the matrix with the $u_j$ as its columns. Otherwise, you’ll have to use your favorite method of constructing a basis for the space $U$ and then apply one of the above methods. Depending on what you have, it might be less work in the general case to compute a basis for $U^\perp$, which is the null space of the matrix $U$, and use that basis to construct $P_\perp$ directly. You can, of course, use the construction given by mathreadler in his answer, but that requires having bases for both $U$ and $U^\perp$, so these other methods are usually less work.

As for your last question, if you have any linear transformation matrix at all, you can always “bulk process” a set of vectors by packing them into the columns of a matrix and multiplying that by your transformation matrix. Each column of the product is the product of the left-hand matrix with the corresponding column of the matrix on the right. That’s just a basic property of matrix multiplication.

You can build a projection matrix like this:

$${\bf P} = {\bf SDS}^{-1}$$

where $\bf D$ is the diagonal matrix : 1 for the basis vectors to project onto and 0 for the ones to reject.

For example: the projection onto the vector [1,-1] can be done by:

$${\bf S}= \left[\begin{array}{cc}1&1\\-1&1\end{array}\right], {\bf D} = \left[\begin{array}{cc}1&0\\0&0\end{array}\right]$$

As you can see the 1 is the first spot of $\bf D$ just as $[1,-1]^T$ in $\bf S$.

Now $$\bf P= \left[\begin{array}{cc}\color{red}{1}&\color{blue}{1}\\\color{red}{-1}&\color{blue}{1}\end{array}\right]\left[\begin{array}{cc}\color{red}{1}&0\\0&\color{blue}0\end{array}\right] \left[\begin{array}{cc}1&1\\-1&1\end{array}\right]^{-1} = \left[\begin{array}{cc}0.5&-0.5\\-0.5&0.5\end{array}\right]$$

And consequently the rejection matrix $$\bf R = I-P =\left[\begin{array}{cc}1&0\\0&1\end{array}\right]- \left[\begin{array}{cc}0.5&-0.5\\-0.5&0.5\end{array}\right] = \left[\begin{array}{cc}0.5&0.5\\0.5&0.5\end{array}\right]$$

The leftmost column in $\bf S$ is the vector we want to project on, this is encoded by the 1 in the upper left corner of $\bf D$. If it instead was the right column vector we wanted to project on then $\bf D = \left[\begin{array}{cc}0&0\\0&1\end{array}\right]$

• I think I almost have it... how did you create the $S$ matrix? – Paul Terwilliger May 23 '17 at 15:05
• @PaulTerwilliger: I updated to clarify the relation between the columns in $\bf S$ and where 0s and 1s are in $\bf D$. – mathreadler May 23 '17 at 16:36
• Now there is very uhm... colorful matching between diagonal elements and vectors in $\bf S$. – mathreadler May 23 '17 at 21:14