Is $\|(I_m + U^\top D U)^{-1} U^\top D\|_F$ always small? Let $n > m$, let $U$ be an $n\times m$ full-rank matrix with positive elements, and let $D$ be $n \times n$ with diagonal elements in $(0,e^{-2})$. Then, is 
\begin{gather*}
\| (I_m + U^\top D U)^{-1} U^\top D \|_F
\end{gather*}
always 'small' ($<1$)? And if so, can one bound this norm, in terms of the elements of $U$ and $D$?
I generated many (100,000,000+) instances with various $m\in\{2,\dots,5\}$ and $n\in\{5,\dots,200\}$ and the elements of $U$ in various intervals, but I was never able to find a norm larger than 1. 
For $m=2$ I was able to derive the following
\begin{align*}
%
\| (I_m + U^\top D U)^{-1} U^\top D \|^2_F &\leq \| (I_m + U^\top D U)^{-1} \|^2_F \cdot \| U^\top D \|^2_F \\
%
&= (u_1^\top D^2 u_1 + u_2^\top D^2 u_2) \cdot \\
& \frac{1 + 2(u_1^\top D u_1 + u_2^\top D u_2) + (u_1^\top D u_1)^2 + (u_2^\top D u_2)^2 + 2(u_1^\top D u_2)^2}{\left(1 + u_1^\top D u_1 + u_2^\top D u_2 + u_1^\top D u_1 u_2^\top D u_2  - (u_1^\top D u_2)^2 \right)^2}
%
\end{align*}
and
\begin{align*}
%
\| (I_m + U^\top D U)^{-1} U^\top D \|^2_F &= \frac{u_1^\top D^2 u_1 + u_2^\top D^2 u_2 + \text{tr}\left( DUR^\top U^\top D U \left( 2  + U^\top D U  \right)RU^\top D \right)}{\left(1 + u_1^\top D u_1 + u_2^\top D u_2 + u_1^\top D u_1 u_2^\top D u_2  - (u_1^\top D u_2)^2  \right)^2} 
%
\end{align*}
where $u_1$ and $u_2$ are the two columns of $U$ (of length $n$) and $R$ is the rotation matrix
$$ R =
\begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix} $$
but I was not able to follow up on this.
Any comments, directions, or counter-examples are very much appreciated!
 A: Let $V:=D^{1/2}U$. Then
$$
(I+U^TDU)^{-1}U^TD=(I+V^TV)^{-1}V^TD^{1/2}.
$$
Now consider the full SVD of $V=PSQ^T$ (full in the sense that $P$ and $Q$ are square orthogonal matrices and $S$ has the same shape as $V$ with the singular values $\sigma_1,\ldots\sigma_m$ on the diagonal).
Then
$$
(I+V^TV)^{-1}V^T=Q(I+S^TS)^{-1}S^TP^T.
$$
For the spectral norm (not Frobenius), we have
$$
\|(I+V^TV)^{-1}V^T\|_2=\|(I+S^TS)^{-1}S^T\|_2=\max_{i=1,\ldots,m}\frac{\sigma_i}{1+\sigma_i^2}\leq\frac{1}{2}.
$$
For the whole matrix, we have hence
$$
\|(I+U^TDU)^{-1}U^TD\|_2\leq\|(I+U^TDU)^{-1}U^TD^{1/2}\|_2\|D^{1/2}\|_2
\leq\frac{1}{2e},
$$
where I've used the assumption on the diagonal matrix $D$ implying that $\|D^{1/2}\|_2\leq 1/e$.
To get a bound on the Frobenius norm, we can use the matrix norm equivalence and get
$$
\|(I+U^TDU)^{-1}U^TD\|_F\leq \sqrt{m}\|(I+U^TDU)^{-1}U^TD\|_2\leq\frac{\sqrt{m}}{2e}.
$$
Note that for your test data ($m\leq 5$), you will never see the Frobenius norm larger than one because
$$
0.411\approx \frac{\sqrt{5}}{2e} < 1.
$$
So yes, the norm is always small or, more precisely, bounded by something supposedly small, but is not generally smaller than one. The following Matlab code is likely to give you a counter-example in the first hit:
n = 200; m = 100;
U = rand(n, m); D = diag(rand(n,1) * exp(-2));
disp(norm((eye(m) + U' * D * U) \ (U' * D), 'fro'));

