show that an orthogonal matrix $Q$ that maximizes $f(Q)=tr(QB)$ satisfies $ Q = \sqrt{B^{\top}B}^{-1}B^{\top} = B^{\top}\sqrt{BB^{\top}}^{-1}$ I can't solve the following problem.
In this problem, matrices are supposed to be over the real numers.
Let $A$ be a symmetric positive definite matrix of order $n$ and $B$ be a nonsingular matrix of order $n$. there exists a unique symmetric positive definite matrix $R$ such that $R^2=A$. We denote such R by $\sqrt{A}$.
Show that an orthogonal matrix $Q$ that maximizes $f(Q)=tr(QB)$ satisfies
$$ Q = \sqrt{B^{\top}B}^{-1}B^{\top} = B^{\top}\sqrt{BB^{\top}}^{-1}.$$
 A: Let $B = \sum_{i=1}^{n} \sigma_i u_i v_i^T $ be the singular value decomposition of $B$.
For any orthogonal $Q$ we have,
$$QB  = \sum_{i=1}^{n} \sigma_i Qu_i v_i^T ,$$ and hence $$
\begin{align}
\text{trace}(QB) &= \sum_{i=1}^{n} \sigma_i \text{trace}( (Qu_i) v_i^T)\\
& = \sum_{i=1}^{n} \sigma_i v_i^TQu_i, \end{align}$$
where we have used $\text{trace} AB = \text{trace} BA.$
By Cauchy-Schwarz for any vectors $a$ and $b$ we have $a^Tb \leq ||a|| ||b||$ so
$$\text{trace}(QB) \leq \sum_{i=1}^{n}\sigma_i ||v_i||||Qu_i|| = \sum_{i=1}^{n}\sigma_i.$$
We can easily check the provided $Q$'s are maximizers.
If $Q = (B^TB)^{-\frac{1}{2}} B^T $ then $QB = (B^TB)^{\frac{1}{2}}.$
Now $B^TB = \sum_{i=1}^{n} \sigma_i^2 v_i v_i^T,$  from which we get $(B^TB)^{\frac{1}{2}} = \sum_{i=1}^{n} \sigma_i v_i v_i^T$ and $\text{trace}(QB) = \sum_i \sigma_i \text{trace}(v_i^Tv_i) = \sum_i \sigma_i .$
We also need to show that $Q$ is orthogonal. 
It follows from $Q^TQ = B (B^TB)^{-\frac{1}{2}} (B^TB)^{-\frac{1}{2}} B^T = B(B^TB)^{-1}B^T = I.$
If $Q=B^T(BB^T)^{-\frac{1}{2}}$, then $BQ = BB^T(BB^T)^{-\frac{1}{2}} = (BB^T)^{\frac{1}{2}}.$
Similar to above $(BB^T)^{\frac{1}{2}} = \sum_i \sigma_i u_i u_i^T$ and  $\text{trace}(QB) = \text{trace}(BQ) = \sum_{i} \sigma_i \text{trace}(u_i^T u_i) = \sum_i \sigma_i.$ Orthogonality proof is also similar.
A: Let $B=USV^T$ be the SVD of $B$. Then, we have
$$ tr(Q B) = tr(V^T Q U S) = \sum_{i=1}^n p_i s_i \le  \max_i |p_i| \sum_{i=1}^n s_i, $$
where $s_i$'s are the singular values of $B$ and $p_i$'s are the diagonal elements of $V^T Q U$. 
Let $u_i, v_i$ be the $i$ th column of $U$ and $V$. By Cauchy Schwarz it follows
$$ |p_i| = | v_i^T Q u_i | \le \|v_i\| \|Qu_i\| = 1$$
for the Euclidean norm. 
Hence, the upper bound is attained by $Q=VU^T$. That is, it is a maximizer. 
Now, you can easily verify that this $Q$ is the same $Q$ as in the question:
$$ \sqrt{ B^T B }^{-1} B^T = \sqrt{ VSSV^T }^{-1} B^T = VS^{-1} V^T V S U^T = VU^T = Q. $$
