Assume $\vec{x}$ and $\vec{y}$ are vectors. We can obtain the projection of vector $\vec{x}$ onto $\vec{y}$ by using the following formula:

$proj_{\vec{x}->\vec{y}} = \frac{\vec{x}\cdot \vec{y}}{\vec{y} \cdot \vec{y}}\vec{y}$

And the rejection of vector $\vec{x}$ onto $\vec{y}$ is

$rejection_{\vec{x}->\vec{y}} = \vec{x} - proj_{\vec{x}->\vec{y}} = \vec{x} - \frac{\vec{x}\cdot \vec{y}}{\vec{y} \cdot \vec{y}}\vec{y}$

Is there a matrix generalization of vector rejection? To say it in more detail:

Assume that $\vec{x}$ is a vector and that the matrix $Y$ represents multiple vectors $\vec{y}_i$. Is there an easy way to calculate the vector rejection of $\vec{x}$ from each vector in $Y$ such that the resulting $\vec{x}$ vector is orthogonal to all vectors $\vec{y}$ in $Y$?

Assume that the matrix $X$ represents multiple vectors $\vec{x}_i$ and that the matrix $Y$ represents multiple vectors $\vec{y}_i$. Is there an easy way to calculate the vector rejection of each vector $\vec{x}_i$ from each vector $\vec{y}_i$ such that each vector $\vec{x}_i$ in $X$ is orthogonal to each vector $\vec{y}_i$ in $Y$?

  • 2
    $\begingroup$ A projection can be represented with a matrix $\bf P$, the rejection is then done by the matrix $({\bf I-P})$ where $\bf I$ is the unit matrix which leaves vectors alone when you multiply with it. $\endgroup$ – mathreadler May 23 '17 at 13:49
  • $\begingroup$ @mathreadler Ok, so how do I find $P$? $\endgroup$ – Paul Terwilliger May 23 '17 at 13:59

If the matrix $P$ represents orthogonal projection onto $U=\operatorname{span}\{u_1,\dots,u_k\}$, then $P_\perp=I-P$ represents represents orthogonal rejection, which is the same as orthogonal projection onto $V^\perp$. We have $PP_\perp v=P(I-P)v=Pv-P^2v=Pv-Pv=0$, so $P_\perp v$ is orthogonal to $U$, and obviously $v=Pv+P_\perp v$.

There are several ways to construct an orthogonal projection matrix $P$, depending on what you’re starting with. If $u_1,\dots,u_k$ form an orthogonal basis for $U$, then you can simply add up the $k$ individual projection matrices. If they’re not orthogonal but are linearly independent, you can use the formula $P=U(U^TU)^{-1}U^T$, where (with a bit of abuse of notation) $U$ is the matrix with the $u_j$ as its columns. Otherwise, you’ll have to use your favorite method of constructing a basis for the space $U$ and then apply one of the above methods. Depending on what you have, it might be less work in the general case to compute a basis for $U^\perp$, which is the null space of the matrix $U$, and use that basis to construct $P_\perp$ directly. You can, of course, use the construction given by mathreadler in his answer, but that requires having bases for both $U$ and $U^\perp$, so these other methods are usually less work.

As for your last question, if you have any linear transformation matrix at all, you can always “bulk process” a set of vectors by packing them into the columns of a matrix and multiplying that by your transformation matrix. Each column of the product is the product of the left-hand matrix with the corresponding column of the matrix on the right. That’s just a basic property of matrix multiplication.


You can build a projection matrix like this:

$${\bf P} = {\bf SDS}^{-1}$$

where $\bf D$ is the diagonal matrix : 1 for the basis vectors to project onto and 0 for the ones to reject.

For example: the projection onto the vector [1,-1] can be done by:

$${\bf S}= \left[\begin{array}{cc}1&1\\-1&1\end{array}\right], {\bf D} = \left[\begin{array}{cc}1&0\\0&0\end{array}\right]$$

As you can see the 1 is the first spot of $\bf D$ just as $[1,-1]^T$ in $\bf S$.

Now $$\bf P= \left[\begin{array}{cc}\color{red}{1}&\color{blue}{1}\\\color{red}{-1}&\color{blue}{1}\end{array}\right]\left[\begin{array}{cc}\color{red}{1}&0\\0&\color{blue}0\end{array}\right] \left[\begin{array}{cc}1&1\\-1&1\end{array}\right]^{-1} = \left[\begin{array}{cc}0.5&-0.5\\-0.5&0.5\end{array}\right]$$

And consequently the rejection matrix $$\bf R = I-P =\left[\begin{array}{cc}1&0\\0&1\end{array}\right]- \left[\begin{array}{cc}0.5&-0.5\\-0.5&0.5\end{array}\right] = \left[\begin{array}{cc}0.5&0.5\\0.5&0.5\end{array}\right]$$

The leftmost column in $\bf S$ is the vector we want to project on, this is encoded by the 1 in the upper left corner of $\bf D$. If it instead was the right column vector we wanted to project on then $\bf D = \left[\begin{array}{cc}0&0\\0&1\end{array}\right]$

  • $\begingroup$ I think I almost have it... how did you create the $S$ matrix? $\endgroup$ – Paul Terwilliger May 23 '17 at 15:05
  • $\begingroup$ @PaulTerwilliger: I updated to clarify the relation between the columns in $\bf S$ and where 0s and 1s are in $\bf D$. $\endgroup$ – mathreadler May 23 '17 at 16:36
  • $\begingroup$ Now there is very uhm... colorful matching between diagonal elements and vectors in $\bf S$. $\endgroup$ – mathreadler May 23 '17 at 21:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.