Does this rectangular matrix exist? If $\mathbf{ABD=0}$, $\mathbf{A}$ and $\mathbf{B}$ are known and $\mathbf{A}$ is $M \times N$, $\mathbf{B}$ is $N \times R$, $\mathbf{B^HB=I}$ and $\mathbf{D}$ is $R \times S$, $R \neq S$. Does $\mathbf{D}$ exist such that $\mathbf{D^HD=I}$ where $\mathbf{H}$ is the conjugate transpose. $\mathbf{BD}$ is generally known, it can be computed as the null matrix for $\mathbf{A}$, but I'm more interested in the solution the gives $\mathbf{D^HD=I, }$.
 A: Edit: the following lengthy discussion aims at proving that there are solutions if and only if $S\leq R$ and $\mbox{rank}AB\leq R-S$. In this case the solution set is the set of isometries from $\mathbb{C}^S$ to $\mbox{Ker}AB$.
The condition $D^*D=I_S$ you want simply means that $D:\mathbb{C}^S\longrightarrow\mathbb{C}^R$ is an isometry. So its range has dimension $S$ and therefore we must have $S\leq R$ is such a $D$ exists. If $S=R$, then $D$ is invertible, cf. previous question. So we can assume $S<R$.
The equation $ABD=0$ is equivalent to $\mbox{Im}D\subseteq \mbox{Ker}AB$.
So your problem has a solution if and only if 
$$
\dim  \mbox{Ker}AB\geq S.
$$
By the rank-nullity theorem, $\dim  \mbox{Ker}AB=R-\mbox{rank}AB$. So this condition is equivalent to
$$
R-\mbox{rank}AB\geq S\quad\iff\quad \mbox{rank}AB\leq R-S.
$$ 
In this case, the solution set is the set of all isometries $D:\mathbb{C}^S\longrightarrow \mbox{Ker}AB$. 
Now given your other assumptions, $B:\mathbb{C}^R\longrightarrow\mathbb{C}^N$ is an isometry. So $R\leq N$ and $B$ has rank $R$. Since $\mbox{Im}AB=A(\mbox{Im}B)$, it follows that 
$$\mbox{rank}AB\leq \min\{R,M\}.$$
Since $A:\mathbb{C}^N\longrightarrow\mathbb{C}^M$ is arbitrary, this rank could be anything from $0$ to $\min\{R,M\}$. So the condition $\mbox{rank}AB\leq R-S$ is not always fulfilled. It depends on $A,B$.
