first of all sorry if the title is not really specific but I thought about it and I didn't find something short which fits better my case.

My problem is the following.

I have a matrix $A\in\mathbb{R}^{12\times12}$ and I have a possible analytical expression of its Null-Space $\mathcal{N}(A)$ (The analytical expression is one of the infinite possible expressions).

I arrived to the point in which I can write:

$\mathcal{N}(A)=span\{ \mathbf{n_1}, \mathbf{n_2}, \mathbf{n_3} \}$


  1. $\mathcal{N}(A)$ is a 5-dimensional subspace of $\mathbb{R}^{12}$ and the analytic expression $\{ \mathbf{n_1}, \mathbf{n_2}, \mathbf{n_3} \}$ is one (of the infinite) basis of the subspace of dimension 5.

Indeed I have:

  1. $\mathbf{n_1}\in\mathbb{R}^{12\times3}$
  2. $\mathbf{n_2}\in\mathbb{R}^{12\times1}$
  3. $\mathbf{n_3}\in\mathbb{R}^{12\times1}$

A possible basis of $\mathcal{N}(A)$ is the following:

$\begin{split} \mathcal{N}(A)&= span\left\{\left[\begin{array}{c} \mathbf{1_{N_3}} \\ \mathbf{0_{3\times3}}\end{array} \right],\, \left[\begin{array}{c} \mathbf{p} \\ \mathbf{0_{3\times1}}\end{array} \right], \left[\begin{array}{c} \mathbf{p}^{\perp} \\ \mathbf{1_N}\end{array} \right] \right\}\\ = span \left\{\left[ \begin{array}{c} 1\\0\\0\\1\\0\\0\\1\\0\\0\\0\\0\\0 \end{array} \begin{array}{c} 0\\1\\0\\0\\1\\0\\0\\1\\0\\0\\0\\0 \end{array} \begin{array}{c} 0\\0\\1\\0\\0\\1\\0\\0\\1\\0\\0\\0 \end{array} \right],\, \left[ \begin{array}{c} p_{1_x}\\ p_{1_y}\\ p_{1_z}\\ p_{2_x}\\ p_{2_y}\\ p_{2_z}\\ p_{3_x}\\ p_{3_y}\\ p_{3_z}\\ 0\\0\\0 \end{array} \right], \left[ \begin{array}{c} p^{\perp}_{1_x}\\ p^{\perp}_{1_y}\\ p^{\perp}_{1_z}\\ p^{\perp}_{2_x}\\ p^{\perp}_{2_y}\\ p^{\perp}_{2_z}\\ p^{\perp}_{3_x}\\ p^{\perp}_{3_y}\\ p^{\perp}_{3_z}\\ 1\\1\\1 \end{array} \right] \right\}\\ &=span\left\{\mathbf{n_1}, \mathbf{n_2}, \mathbf{n_3}\right\} \end{split}$

Where: $\mathbf{1_{N_3}}=\left[\begin{array}{c} 1\\1\\1 \end{array} \right]\otimes\mathbf{I}_{3\times3}\in\mathbb{R}^{9\times3}$

And $\mathbf{p}$ is the vector of positions of certain agents (it doesn't matter here I think ($\mathbf{N}$ is the number of agents, in this case $\mathbf{N}=3$).

$\mathbf{p}=\left[\begin{array}{c} \mathbf{p_{1_x}}\\\mathbf{p_{1_y}}\\ \mathbf{p_{1_z}} \\ \mathbf{p_{2_x}}\\\mathbf{p_{2_y}}\\ \mathbf{p_{2_z}} \\ \mathbf{p_{3_x}}\\\mathbf{p_{3_y}}\\ \mathbf{p_{3_z}} \end{array} \right]$

I want to find the orthogonal complement of $\mathcal{N}(A)$.

In order to do this I can first find $\mathbf{n_1^{\perp}}$ which is $\in\mathbb{R}^{12\times9}$. The problem is that $\mathbf{n_1^{\perp}}$ spans already the subspace spanned by $\mathbf{n_2},\mathbf{n_3}$. This means that the set $S=\{\mathbf{n_1^{\perp}}, \mathbf{n_2},\mathbf{n_3}\}$ is linear dependent. This is normal also from a logical point of view because it is like we are in $\mathbb{R}^{12}$ and because $\mathbf{n_1}\in\mathbb{R}^{12\times3}$ and $\mathbf{n_1^{\perp}}$ is $\in\mathbb{R}^{12\times9}$ we already covered all the $(3+9)=12$ dimensions. This can also be seen in this way:


But I said that I want to find the orthogonal complement of $\mathcal{N}(A)$ and for sure the subspace spanned by $\mathbf{n_1^{\perp}}$ contains already the subsubspace spanned by $\mathbf{n_2,n_3}$ so my question is:

how can I find a subspace of $\mathbf{n_1^{\perp}}$ which doesn't contain the subspace spanned by $\mathbf{n_2}, \mathbf{n_3}$?

I hope I was clear and thanks a lot for your time.

  • 1
    $\begingroup$ Your notation is unclear. I would expressions like $\Bbb R^{12\times5}$ as denoting a vector space of rectangular matrices, not a as subspace of some $\Bbb R^n$ which $\mathcal N(A)$ would be. Also you didn't say what size $A$ is, so we don't even know which $n$ to take there. Finally, subspaces are not finite sets of vectors, so the notation $\{n_1,n_2,n_3\}$ is not appropriate (maybe you meant the span of such a set?). $\endgroup$ – Marc van Leeuwen Dec 9 '15 at 13:47
  • $\begingroup$ thanks for the quick comment. I will add the size of matrix $A$ but the Null-space of the matrix is a collection of vectors and that is why i put it to be $\in \mathbb{R}^{12\times5}$ $\endgroup$ – kalmanIsAGameChanger Dec 9 '15 at 13:49
  • 1
    $\begingroup$ Well, a nulll space is an infiinte set of vectors, so writing $\Bbb R^{12\times5}$ for that is not appropriate. If you mean a $5$-dimensional subspace of $\Bbb R^{12}$ then please say so explicitely. Such a space could be described by $5$ vectors spanning it, but such a set is far from unique, so you should not identify $\mathcal N(A)$ with such a collection of vectors. $\endgroup$ – Marc van Leeuwen Dec 9 '15 at 13:52
  • $\begingroup$ Ok sorry, I didn't understand what you meant. My fault. I tried to correct the question. Let me know what you think about it now. $\endgroup$ – kalmanIsAGameChanger Dec 9 '15 at 14:50
  • 1
    $\begingroup$ $\mathbf n_1$ appears to be a matrix. How can that be an element of a basis for a vector space? $\endgroup$ – amd Dec 9 '15 at 20:13

If you have a set of generators $W_i$ for a subspace $W$ of a vector space, then the orthogonal complement $W^\perp$ is the intersection of the $W_i^\perp$. It doesn’t matter if any of those orthogonal complements include some of the generators, since that will all get sorted out when you take their intersection. However, there are several ways to find $W^\perp$ without resorting to intersecting vector spaces. Here are a few of the more common ones.

If you have a set of vectors $\{\mathbf w_i\}$ that generates $W$, then for any $\mathbf v\in W^\perp$, $\mathbf w_i\cdot\mathbf v=0$ for all $i$, i.e., $W^\perp$ is the solution set to the homogeneous system of linear equations $A\mathbf v=0$, where the matrix $A$ has as its rows the $\mathbf w_i$.

Other ways involve constructing the orthogonal projection operator onto $W$, $\pi_W$. Then, $\ker\pi_W = \operatorname{im}(1-\pi_W)=W^\perp$, so the problem becomes one of finding the kernel or image of a linear map, which is widely documented elsewhere.

There are several ways to construct a matrix for $\pi_W$. If you have an orthogonal basis $\{\mathbf u_i\}$ for $W$, then you can construct $\pi_W$ as the sum of individual orthogonal projections onto the $\mathbf u_i$: $\sum_i{\mathbf u_i\mathbf u_i^T \over \mathbf u_i\mathbf u_i^T} = \sum_i{\mathbf u_i\mathbf u_i^T \over \mathbf u_i^T\mathbf u_i}$. A similar sum holds for a decomposition of $W$ into a direct sum of mutually orthogonal spaces $W_1\oplus W_2\oplus\cdots\oplus W_k$: $\pi_W = \sum_i\pi_{W_i}$. This is really only useful if you can find the individual projections some other way.

You often have a basis $\{\mathbf u_i\}$ for $W$, but it’s not orthogonal. The Gram-Schmidt process will produce an orthogonal (orthonormal) basis from it, but it’s not necessary to go through that. If you have any basis—not necessarily orthogonal—for $W$, then another fairly easy-to-compute expression for $\pi_W$ is $\pi_W=U(U^TU)^{-1}U^T$, where $U$ has as its columns the vectors $u_i$.

If you’re not using the Euclidean scalar product to define orthogonality, these formulas will need to be modified accordingly. Let $Z$ be the matrix associated with this scalar product, i.e., $\langle\mathbf u,\mathbf v\rangle_Z=\mathbf v^TZ\mathbf u$. Then orthogonal projection onto a vector $\mathbf u$ is given by ${\mathbf u\mathbf u^T \over \mathbf u^TZ\mathbf u}Z$, and the last expression for $\pi_W$ becomes $U(U^TZU)^{-1}U^TZ$. The system of linear equations in the first method would also change to $AZ\mathbf v=0$.

  • 1
    $\begingroup$ @minidiable A basis for a 5-dimensional space consists of five vectors. You have two vectors and what appears to be a matrix. Are the columns of that matrix meant to be the other vectors that form your basis? $\endgroup$ – amd Dec 10 '15 at 10:10
  • 1
    $\begingroup$ Your notation still obfuscates things for me. I’ve added a few other options to my answer, but after this I don’t expect that I’ll work on this problem any more. $\endgroup$ – amd Dec 11 '15 at 8:21
  • 1
    $\begingroup$ @minidiable For me, at least, if you expanded $\mathbf n_1$ into individual vectors, that would make this clearer. I’ll leave you with one other possibility: since $\operatorname{im}A^*=\ker A^\perp$, if there’s no other reason to compute $A$’s kernel, it might be more efficient to work directly with its adjoint. $\endgroup$ – amd Dec 11 '15 at 20:58
  • 1
    $\begingroup$ @minidiable You’ve got it. Once you have $\pi_W$, its kernel will give you the orthogonal complement you seek. As to your other point, if the $\mathbf n_i$ aren’t mutually orthogonal, then there’s no guarantee that their individual orthogonal complements will also be orthogonal to the other parts of $\mathcal N(A)$. In that case, you would need to find the intersection of these orthogonal complements to pick out those vectors orthogonal to all: $W^\perp=\mathbf n_1^\perp\cap\mathbf n_2^\perp\cap\mathbf n_3^\perp$. $\endgroup$ – amd Dec 16 '15 at 19:06
  • 1
    $\begingroup$ @minidiable Now that I think about it, if the $\mathbf n_i$ are mutually orthogonal, then you’re guaranteed that each $\mathbf n_i^\perp$ will contain all of the other $\mathbf n_j$ $(i\ne j)$. If you go that route, you’ll have to compute the intersection of all of the individual orthogonal complements somehow whether or not the $\mathbf n_i$ are orthogonal. $\endgroup$ – amd Dec 17 '15 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.