Characterization of unitarily equivalent direct sum of irreducible operators Let $(\mathscr{H_i})_{i\in I}$ be a collection Hilbert spaces and define $\mathscr{H} = \{h:I\rightarrow \cup_i\mathscr{H}_i:h(i)\in \mathscr{H}_i,\  \sum_{i\in I}\|h(i)\|^2<\infty\}$. Then it is easy to show that $\mathscr{H}$ is a Hilbert space. If we have uniformly bounded operators $\{A_i\}_{i\in I}$ on $\mathscr{H}_i$ we define $(Ah)(i)=\Big[(\oplus_j A_j)h\Big](i) = A_ih(i)$ so that $A$ is a bounded operator on $\mathscr{H}$. My question is now the following

Suppose $A = \oplus_{i\in I}A_i$ and $B = \oplus_{i\in I}B_i$ and that each $A_i$ and $B_i$ is irreducible (meaning that the only spaces so that the space together with its orthogonal complement is reducible under $A_i$ is $\mathscr{H_i}$ and $\{0\}$). Show that $A\simeq B$ if and only if we can find a permutation $\pi:I\rightarrow I$ such that $A_i\simeq B_{\pi(i)}$.

Here $A\simeq B$ means that we can find an isomorphism $U$ such that $UAU^{-1} = B$. Now I can prove the direction $\Leftarrow$ without too much difficulty however it is the $\Rightarrow$ direction which I have trouble with. My reasoning so far is along the following lines:
Let $\hat{\mathscr{H}}_j = \{h\in \mathscr{H}: h(i) = 0\text{ if $i\neq j$}\}$. Then it is clear that $B\hat{\mathscr{H}}_j\subseteq \mathscr{H}_j$ by the definition of $B$. Thus if $A\simeq B$ we conclude that
$$UAU^{-1}\hat{\mathscr{H}}_j\subseteq \hat{\mathscr{H}}_j\Rightarrow AU^{-1}\hat{\mathscr{H}}_j\subseteq U^{-1}\hat{\mathscr{H}}_j.$$
Now if $P_i:{\mathscr{H}}\rightarrow {\mathscr{H}}$ is defined by $P_ih = h(i)$ then $P_iA = A_iP_i$ and therefore
$$A_iP_iU^{-1}\hat{\mathscr{H}}_j = P_i AU^{-1}\hat{\mathscr{H}}_j\subseteq P_iU^{-1}\hat{\mathscr{H}}_j.$$
By applying a similar reasoning to $A^\ast = \oplus A_i^\ast$ one may conclude that $P_iU^{-1}\hat{\mathscr{H}}_j$ reduces $A_i$ and therefore if it is non-zero then $P_iU^{-1}\hat{\mathscr{H}}_j = \mathscr{H}_i$.
Now I want to use this to conclude that $\hat{\mathscr{H}}_i\subseteq U^{-1}\hat{\mathscr{H}}_j$ however I have trouble completing this argument. If I could show this then I could argue that 
$$U^{-1}BU\hat{\mathscr{H}}_i =A\hat{\mathscr{H}}_i\subseteq \hat{\mathscr{H}}_i$$
why $$BU\hat{\mathscr{H}}_i\subseteq U\hat{\mathscr{H}}_i\subseteq \hat{\mathscr{H}}_j$$
and therefore
$$B_jP_jU\hat{\mathscr{H}}_i\subseteq P_jU\hat{\mathscr{H}}_i$$
so that $U\hat{\mathscr{H}}_j = \hat{\mathscr{H}}_i = \mathscr{H}_j$
and from this I could build my isomorphism. However I don't know how to get to this point.
 A: Here is a proof of the forward direction, using an additional fact which requires the spectral theorem or at least functional calculus; see the comments at https://math.stackexchange.com/a/1714909/467147 (noting that a self-adjoint operator commutes with $A$ if and only if it commutes with the star-algebra generated by $A$).

Fact. If $A:H\to H$ is irreducible then the only self-adjoint operators that commute with $A$ are scalar multiples of the identity operator.

I'll also assume that $\dim \mathscr{H}_i\neq 0$ - I don't know if this is important here but it is a generally good convention that the unique map $\mathbb C^0\to \mathbb C^0$ is not considered irreducible.
Fix a unitary $U$ such that
$$UA=BU.$$
For $i,j\in I$ let $\iota_j$ be the direct sum inclusion $\mathscr{H}_j\to\mathscr{H}$ and $\pi_i$ he direct sum projection $\mathscr{H}\to\mathscr{H}_i.$ So $\iota_i$ is the adjoint of $\pi_i.$ Define $U_{i,j}:\mathscr{H}_j\to\mathscr{H}_i$ by
$U_{i,j}=\pi_i U\iota_j.$ I imagine $U_{i,j}$ as the $(i,j)$ element of $U,$ as a kind of block matrix.
Note
$$U_{i,j}A_j=\pi_iU\iota_jA_j=\pi_iUA\iota_j=\pi_iBU\iota_j=B_i\pi_iU\iota_j=B_jU_{i,j}.$$
Also, $UA=BU$ implies $UA^*=B^*U,$ so we similarly have
$$U_{i,j}A_j^*=B_j^*U_{i,j}$$
So:
$$U_{i,j}^*U_{i,j}A_j=A_jU_{i,j}^*U_{i,j}$$
$$U_{i,j}U_{i,j}^*B_i=B_iU_{i,j}U_{i,j}^*$$
Using the Fact above, there are $\lambda,\lambda'$ such that $U_{i,j}^*U_{i,j}=\lambda\cdot  {\mathrm{id}}_j$ and $U_{i,j}U_{i,j}^*=\lambda'\cdot  {\mathrm{id}}_i,$ where $\mathrm{id}_i$ is the identity operator $\mathscr{H}_i\to\mathscr{H}_i.$ Contemplating $(U_{i,j}^*U_{i,j})(U_{i,j}^*U_{i,j})=U_{i,j}^*(U_{i,j}U_{i,j}^*)U_{i,j}$ proves $\lambda^2=\lambda\lambda',$ and swapping $U_{i,j}$ with $U_{i,j}^*$ gives similarly $\lambda'^2=\lambda\lambda',$ and together these give $\lambda=\lambda'.$ Call this common value $\lambda_{i,j}.$ It must be non-negative because $U_{i,j}^*U_{i,j}$ is a positive operator ($\lambda\langle v,v\rangle=\langle U_{i,j}v,U_{i,j}v\rangle\geq 0$ for any vector $v\in\mathscr{H}_j$).
The matrix $(\lambda_{i,j})_{i,j\in I}$ is doubly stochastic: its entries are non-negative, and $\sum_i \lambda_{i,j}=1$ for each $i\in I$ because $\sum_i U_{i,j}^*U_{i,j}=\mathrm{id}_j,$ and similarly $\sum_j\lambda_{i,j}=1$ for each $j\in I.$
If $\lambda_{i,j}\neq 0$ then $U_{i,j}/\sqrt{\lambda}$ is an isomorphism $\mathscr{H}_j\to\mathscr{H}_i$ witnessing that $A_i\simeq B_j.$ 
This means that the terms in $\sum_i\lambda_{i,j}$ can only be non-zero for $j$ such that $B_j\simeq A_i,$ and similarly for $\sum_j\lambda_{i,j}.$ For each $\simeq$-isomorphism class $\mathbf C,$ define $I_{\mathbf C}=\{i\in I\mid A_i\in\mathbf C\}$ and $J_{\mathbf C}=\{j\in I\mid B_j\in\mathbf C\}.$ Then the $I_{\mathbf C}\times J_{\mathbf C}$ submatrix of $(\lambda_{i,j})_{i,j}$ must be doubly stochastic. The following lemma then shows that $I_{\mathbf C}$ and $J_{\mathbf C}$ are in bijection, and combining these bijections over all $\mathbf C$ gives the required permutation of $I.$

Lemma. For a doubly stochastic matrix $(\lambda_{i,j})_{i\in I,j\in J}$ with rows indexed by $I$ and columns indexed by $J,$ we have $|I|=|J|.$

Proof. By symmetry it suffices to show $|I|\leq |J|.$ If $J$ is finite then $\sum_{j\in J}\sum_{i\in I} \lambda_{i,j}=|J|,$ so $\sum_{i\in I} \sum_{j\in J}\lambda_{i,j}=|J|,$ which implies $|I|=|J|.$ Otherwise, the cardinality of the set of pairs $(i,j)$ such that $\lambda_{i,j}\neq 0$ is at least $|I|$ and at most $|J\times\aleph_0|=|J|,$ so $|I|\leq |J|.$ $\square$
