Considering operators on the direct sum of Hilbert spaces as operator valued matrices If $(\mathcal H_i)_{i \in I}$ is a family of Hilbert spaces, we can form their Hilbert space direct sum
$$
\tilde{\mathcal H} := \oplus^{\ell^2}_{i \in I} \mathcal H_i
$$
where the $\ell^2$ expresses that norms of elements of $\tilde{\mathcal H}$ have to summable with respect to the 2-norm. 
Now, for simplicity, let $\mathcal H_i = \mathcal H$ for one fixed Hilbert space and let
$$
\tilde{\mathcal H} := \oplus^{\ell^2}_{i \in I} \mathcal H = \ell^2(I, \mathcal H),
$$
using the notation of Bochner-Lebesgue spaces (I think I will have to restrict $\mathcal H$ to be a separable space, right?).
Let $\mathfrak B(\mathcal H)$ denote the space of all bounded operators on $\mathcal H$. From the algebraical viewpoint we have that 
$$
\mathfrak B(\ell^2(I, \mathcal H)) \subseteq M_I(\mathfrak B(\mathcal H)) := \left\{ (x_{ij})_{ij} \colon x_{ij} \in \mathfrak B(\mathcal H) \right\},
$$
i.e. every element of $\mathfrak B(\ell^2(I,\mathcal H))$ is a "matrix" with entries in $\mathfrak B(\mathcal H)$.
Can we characterize an element $x$ of $\mathfrak B(\ell^2(I,\mathcal H))$ in terms of its "entries" $x_{ij}$?

Here are my thoughts: Let $x = (x_{ij})_{ij}$ and $\xi = \oplus_i \xi_i \in \tilde{\mathcal H}$. If we want $x \in \mathfrak B(\tilde{\mathcal H})$ the following calculation could provide some necessary conditions:
$$
\| x\xi\|_2^2 = \left\| \bigoplus_i \left(\sum_j x_{ij} \xi_j\right) \right\|_2^2 = \sum_i \left\| \sum_j x_{ij} \xi_j \right\|^2
\leq \\ 
\sum_i \sum_j \|x_{ij}\|^2 \|\xi_j\|^2 
\leq \underbrace{\sum_i \left( \sup_j \|x_{ij}\| \right)^2}_{=: C < \infty} \sum_j \|\xi_j\|^2,
$$
where for the last inequality to hold I assumed that for each $i \in I$ the family $(x_{ij})_{j \in I}$ is uniformly bounded and that the family $s_i := \sup_j \|x_{ij}\|$ is in $\ell^2$.
This would give me something like $x \in \ell^2( I, \ell^\infty(I, \mathfrak B(\mathcal H))$, but this is clearly not an algebra. 
I am happy about comments on my approach. Maybe someone can share a reference on this matter?
 A: *

*I don't think $\mathcal H$ separable gives you a particular advantage here. 

*It is always true that the entries of a bounded operator are uniformly bounded: given $i,j$ you can always find unit vectors such that 
$$
|\langle x_{ij}\xi,\eta\rangle| = |\langle x\tilde\xi,\tilde\eta\rangle|\leq\|x\|,
$$
so $\|x_{ij}\|\leq\|x\|$. So this is a necessary condition. 

*I don't think there is any useful characterization of the entries of a bounded operator. Note that the condition you found is sufficient, not necessary, but it fails on very easy bounded operators like the identity, where $s_i=1$ for all $i$. If even fails for compact operators: in $\mathcal B(\mathcal H)$, you can take $x=\sum_{k}\frac1{\sqrt k}e_{kk}$; this is compact, but the sequence $s_i$ is the harmonic. You can extend this example to any direct sum $\bigoplus_n\mathcal H_n$, by taking $x=\bigoplus_n\frac1{\sqrt n}p_n$, there $p_n\in\mathcal B(\mathcal H_n)$ is a finite-rank projection. In summary, your possible characterization doesn't even cover all compact operators. 

*Note that your approach includes $\mathcal B(\mathcal H)$ itself, where you see $\mathcal H$ as the $\ell^2$-direct sum of $\mathbb C$. 

*I read somewhere, a long time ago, that in the 50s there were serious efforts by some of the pioneers of operator theory/operator algebras to develop a deep treatment of operators as matrices, and this is was basically a great failure. 
