Extended matrix function I have a continuous matrix-valued function $f:\mathbb{R}^d\mapsto {\cal M}_{k\times d}$, with $d<k$, such that $f(x)$ is full rank for all $x\in\mathbb{R}^k$. Can I extend this function to be a continuous function $g:\mathbb{R}^k\mapsto {\cal M}_{k\times k}$ such that $g(x)$ is full rank matrix, the first $d$ columns of $g(x)$ equals to $f(x)$, and the rest columns are orthogonal with all columns of $f(x)$.
Note that since $f(x)$ is full rank then we can add $k-d$ columns such that the obtained matrix is full rank (invertible). Then, using the Gram-Schmidt procedure which is a continuous process we can get $g(x)$ if the $k-d$ vectors that we choose is also conttinuous. But, how can I choose this $k-d$ vectors. Thanks for any help
 A: Extension from $\mathbb{R}^d$ to $\mathbb{R}^k$ can be done by making the function depend on first $d$ coordinates only. So we work with a problem of extending function $f\colon \mathbb{R}^m \to \mathcal{M}_{k\times d}$ into function $g_3\colon \mathbb{R}^m \to \mathcal{M}_{k\times k}$. We need only the case $m=d$, but we will not use that.
First, let's go to polar coordinates: replace $f$ with the function $f_1 \colon \mathbb{R}_+ \times S^{m-1} \to \mathcal{M}_{k\times d}$, defined by
$f_1(\rho,y) = f(\rho y)$. Note, that it is continuous and $f_1(0,y)$ doesn't depend on $y$.
Note, that it is enough to construct a function $g_1\colon \mathbb{R}_+ \times S^{m-1} \to \mathcal{M}_{k\times k}$, which satisfies the following properties:


*

*$g_1$ is continuous;

*$g_1(0,y)$ doesn't depend on $y$;

*$g_1(\rho,y)$ is full-rank;

*$g_1(\rho,y)$ extends $f_1(\rho,y)$ (i.e. first $d$ columns are the same).


Indeed, we then can define $g_2(x) = g_1(\left\|x\right\|,x/\left\|x\right\|)$ for $\left\|x\right\|\neq 0$ and $g_2(0)=g_1(0,y)$ with any $y$
(due to 2-nd property rhs doesn't depend on $y$). Then define $g_3(x)$ with first $d$ columns as in $g_2(x)$,
and last $k-d$ obtained from $g_2(x)$ by Gramm-Schmidt procedure. As you noted in your question, Gramm-Schmidt procedure doesn't break continuity.
Later in the proof we will refer to this procedure simply as "apply Gramm-Schmidt" without repeating all the details above.
In fact, last $k-d$ vectors of $g_1(\rho,y)$ will already be orthogonal to the first $d$, so this paragraph was needed mainly to introduce ``apply Gramm-Schmidt'' procedure.
So we reduced our question to the following. Given continuous function $f_1\colon \mathbb{R}_+ \times S^{m-1} \to \mathcal{M}_{k\times d}$ satisfying properties 1-3 above, can we extend it to a function $g_2\colon \mathbb{R}_+ \times S^{m-1} \to \mathcal{M}_{k\times k}$, satisfying properties 1-4?
This question looks easier to me, because we can start defining $g_2$ for $\rho=0$, and extend this definition to larger value of $\rho$, ensuring that the procedure is continuous in $y$. In other words, we replaced $m$-dimensional variable $x$ with one-dimensional variable $\rho$, thus making problem one-dimensional in a sense. Of course, we still have to ensure, that our actions are continuous with respect to $y$.
Define $f_{-}(\rho,y) = ((f_1(\rho,y))^Tf_1(\rho,y)^{-1}(f_1(\rho,y))^T$ - pseudo-inverse of $f_1(\rho,y)$, satisfying $f_{-}(\rho,y) f_1(\rho,y) = 1_d$.
Define $P_0(\rho,y) = f_1(\rho,y) f_{-}(\rho,y)$ - projector span of columns of $f_1(\rho,y)$, $P_1(\rho,y)=1_k-P_0(\rho,y)$ - projector on the orthogonal complement.
Since we already know first $d$ columns of $g_1$, it is enough to define last $k-d$. We denote the matrix,
consisting of these last $k-d$ columns of $g_1(\rho,y)$ with $h_1(\rho,y)$.
In fact, $k-d$ columns of $h_1(\rho,y)$ will form orthonormal basis in the range of $P_1$. Algebraically, this condition is equivalent to
(a) $P_1h_1(\rho,y)=h_1(\rho,y)$ and (b) $(h_1(\rho,y))^Th_1(\rho,y)=1_{k-d}$.
But suppose instead of $h_1(\rho,y)$ we have $h_0(\rho,y)$, still satisfying $(b)$, but not (a).
If its columns are linearly independent of columns of $f_1(\rho,y)$, we can apply Gramm-Schidt. One can check, that a sufficient condition for this is
$$\operatorname{Tr}\left( (h_0(\rho,y))^T P_0(\rho,y) h_0(\rho,y) \right)<1.$$
We define $g_1(\rho,y)$ (or, equivalently, $h_1(\rho,y)$) by inductive procedure, so that after step $n$ it will be defined for $\rho\in [0,n]$.
Step 0: we want to define $g_1(0,y)$. Notice, that it shouldn't depend on $y$. Good for us, we already have $f_1(0,y)$, which doesn't depend on $y$.
So we have to extend a single matrix. To do this write columns of $f_1(0,y)$ followed by $k$ basis vectors, giving a list of $k+d$ vectors spanning $\mathbb{R}^d$. 
Then go through this list and get rid of all vectors, which can be expressed as a linear combinations of previous ones.
Apply Gramm-Schmidt to the obtained list of $k$ vectors.
Step $n+1$: we assume, that $h_1(\rho,y)$ is already defined for $\rho\in [0,n]$, and now we want to continuously extend this definition to $[0,n+1]$.
Note, that $P_0(\rho,y)$ is continuous with respect to $\rho$ for $\rho\in [n,n+1]$. Therefore it is uniformly continuous (uniformly with respect to both $\rho$ and $y$,
so we can take $\delta=1/N$ (which doesn't depend on $y$),
s.t. $\left\|P_0(\rho_1,y)-P_0(\rho_2,y)\right\|<1/(k-d)$ for $\left\|\rho_1-\rho_2\right\|<1/N$. We then do this step in $N$ substeps, on $l$-th extending
$h_1(\rho,y)$ from $[0,n+(l-1)/N]$ to $[0,n+l/N]$ by first taking $h_0(\rho,y)=h_1(n+(l-1)/N,y)$ and applying Gramm-Schmidt.
Intorducing $\rho_{l-1} = n+(l-1)/N$ note, that we have
\begin{multline*}
\operatorname{Tr}\left( (h_0(\rho,y))^T P_0(\rho,y) h_0(\rho,y) \right)=\\ \operatorname{Tr}\left( (h_1(\rho_{l-1},y))^T P_0(\rho_{l-1},y) h_0(\rho_{l-1},y) \right)+\\ \operatorname{Tr}\left( (h{}_1(\rho_{l-1},y))^T (P_0(\rho,y)-P_0(\rho_{l-1},y)) h_0(\rho_{l-1},y) \right)
\end{multline*}
with first term equal to 0, and second term being less than 1, so Gramm-Schmidt succeeds.
