This answer here is solely for the purpose of giving this question an answer. Since the OP obtained the desired answer (see the comments under the question), I am providing a different way using category theory to show that $$P:=\frac{\prod\limits_{\alpha\in J}\,A_\alpha}{\prod\limits_{\alpha\in J}\,B_\alpha}\cong \prod_{\alpha\in J}\,\frac{A_\alpha}{B_\alpha}\text{ and }S:=\frac{\bigoplus\limits_{\alpha\in J}\,A_\alpha}{\bigoplus\limits_{\alpha\in J}\,B_\alpha}\cong \bigoplus_{\alpha\in J}\,\frac{A_\alpha}{B_\alpha}\,.$$
Explicit isomorphisms can be seen in (*) and (#).
For each $\beta \in J$, $\iota_\beta:A_\beta\to \bigoplus\limits_{\alpha\in J}\,A_\alpha$ and $\pi_\beta: \prod\limits_{\alpha\in J}\,A_\alpha\to A_\beta$ denote the canonical injection and the canonical projection, respectively. Let $q:\bigoplus\limits_{\alpha\in J}\,A_\alpha\to S$ be the quotient map. Then, $q\circ \iota_\beta$ vanishes on $B_\beta$. Therefore, $q\circ \iota_\beta$ factors through the quotient map $q_\beta:A_\beta\to\dfrac{A_\beta}{B_\beta}$. In other words, there exists a (unique) map $i_\beta:\dfrac{A_\beta}{B_\beta}\to S$ such that $$q\circ \iota_\beta=i_\beta\circ q_\beta\,.$$
We claim that $S$ together with the maps $i_\beta:\dfrac{A_\beta}{B_\beta}\to S$ for $\beta\in J$ is a categorical coproduct (direct sum) of the family $\left(\dfrac{A_\alpha}{B_\alpha}\right)_{\alpha\in J}$. Let $T$ be any $R$-module together with morphisms $\tau_\beta:\dfrac{A_\beta}{B_\beta}\to T$ for each $\beta\in J$. We want to show that a there exists a unique morphism $\phi:S\to T$ such that $\phi\circ i_\beta=\tau_\beta$ for each $\beta\in J$.
We define
$$\phi\left((a_\alpha)_{\alpha\in J}+\bigoplus_{\alpha\in J}\,B_\alpha\right):=\sum_{\alpha\in J}\,\tau_\alpha\left(a_\alpha+B_\alpha\right)\text{ for all }(a_\alpha)_{\alpha\in J}\in\bigoplus_{\alpha\in J}\,A_\alpha\,.$$
It is easy to verified that $\phi$ is a well defined morphism, and it is the only morphism such that $\phi\circ i_\beta=\tau_\beta$ for all $\beta\in J$. We can now then conclude that $S$ is a coproduct of the family $\left(\dfrac{A_\alpha}{B_\alpha}\right)_{\alpha\in J}$. Since coproducts are unique up to isomorphism, we obtain $S\cong \bigoplus\limits_{\alpha \in J}\,\dfrac{A_\alpha}{B_\alpha}$, via the isomorphism $\sigma:S\to \bigoplus\limits_{\alpha \in J}\,\dfrac{A_\alpha}{B_\alpha}$ given by
$$\sigma\left((a_\alpha)_{\alpha\in J}+\bigoplus_{\alpha\in J}\,B_\alpha\right):=\sum_{\alpha\in J}\,\bar{\iota}_\alpha\left(a_\alpha+B_\alpha\right)\text{ for all }(a_\alpha)_{\alpha\in J}\in\bigoplus_{\alpha\in J}\,A_\alpha\,,\tag{*}$$
where $\bar{\iota}_\beta:\dfrac{A_\beta}{B_\beta}\to \bigoplus\limits_{\alpha\in J}\,\dfrac{A_\alpha}{B_\alpha}$ is the canonical injection for each $\beta\in J$.
Observe now that, for every $\beta\in J$, $q_\beta\circ \pi_\beta$ vanishes on $\prod\limits_{\alpha\in J}\,B_\alpha$. Therefore, $q_\beta\circ\pi_\beta$ factors through the quotient map $k:\prod\limits_{\alpha\in J}\,A_\alpha\to P$. Ergo, there exists a (unique) morphism $\varpi_\beta:P\to \dfrac{A_\beta}{B_\beta}$ such that $$q_\beta\circ\pi_\beta=\varpi_\beta\circ k\,.$$ We claim that $P$ together with the morphisms $\varpi:P\to \dfrac{A_\beta}{B_\beta}$ is a categorical product (direct product) of the family $\left(\dfrac{A_\alpha}{B_\alpha}\right)_{\alpha\in J}$. Let $Q$ be any $R$-module together with morphisms $\kappa_\beta:Q\to\dfrac{A_\beta}{B_\beta}$ for all $\beta\in J$. We need to show that there exists a unique morphism $\psi:Q\to P$ such that $\varpi_\beta\circ \psi=\kappa_\beta$ for all $\beta\in J$.
We define
$$\psi\left(x\right):=\big(\kappa_\alpha(x)\big)_{\alpha\in J}+\prod_{\alpha\in J}\,B_\alpha\text{ for all }x\in Q\,.$$
It is easily seen that $\psi$ is a well defined morphism, and it is the only morphism such that $\varpi_\beta\circ \psi=\kappa_\beta$ for all $\beta\in J$. We now conclude that $P$ is indeed a product of the family $\left(\dfrac{A_\alpha}{B_\alpha}\right)_{\alpha\in J}$. Since products are unique up to isomorphism, we have $P\cong \prod\limits_{\alpha\in J}\,\dfrac{A_\alpha}{B_\alpha}$ via the isomorphism $\varsigma: \prod\limits_{\alpha\in J}\,\dfrac{A_\alpha}{B_\alpha}\to P$ given by
$$\varsigma\Big(\big(a_\alpha+B_\beta\big)_{\alpha\in J}\Big):=\big(a_\alpha\big)_{\alpha\in J}+\prod_{\alpha\in J}\,B_\alpha\text{ for all }\big(a_\alpha\big)_{\alpha\in J}\in \prod_{\alpha\in J}\,A_\alpha\,.\tag{#}$$