Zeta function identity, formal justification We have the following identity: (where $\mu$ is Mobius function)
$$\frac{\zeta(s)}{\zeta(2s)} = \sum_{n=1}^{\infty} \frac{|\mu(n)|}{n^s} \quad (\sigma > 1, s = \sigma+it )$$
The proof I read (Titchmarsh p5) is given by the following line: 
$$\frac{\zeta (s) }{ \zeta(2s) } \stackrel{*}{=} \prod_p \frac{1-p^{-2s}}{1-p^{-s}} = \prod_p \big( 1 + \frac{1}{p^s} \big) \stackrel{**}{=} \sum_{n=1}^\infty \frac{|\mu(n)|}{n^s}$$

My question is how do we justify $*$ and $**$ rigorously? Below is what I think: 

$(*):$ We note that the partial products, $\hat{f_n} = f_1 \cdots f_n$ where $f_i = \big( 1 - \frac{1}{p_i^s} \big)^{-1}$ converges compactly in region $\sigma >1$. $\zeta(s) = \lim_{n \rightarrow \infty} \hat{f}_n(s)$ (by Euler Product formula). Similarly, $\zeta(2s) = \lim_{n \rightarrow \infty} \hat{g}_n(s)$ where $\hat{g}_n = \hat{f}_n (2s)$. We note that $|\zeta(s)| > 0$ for all $\sigma >1 $. 
$\frac{1}{\zeta(2s)} = \lim_{n \rightarrow \infty} \frac{1}{\hat{g}_n(s)}$ is well-defined, and so by basic algebra of limits, 
$$\frac{\zeta(s) }{ \zeta(2s)} = \lim_{n \rightarrow \infty} \frac{\hat{f}_n(s)} { \hat{g}_n(s) } $$
$(**):$ Here we argue by taking a prime $P$, and we have, 
$$ \Big| \sum \frac{|\mu(n)| }{n^s} - \prod_{p \le P} (1+ \frac{1}{p^s} ) \Big| \le \sum \frac{1}{(P+n)^s} \rightarrow 0 $$
as $P \rightarrow \infty$.

In general, when do we know we can just expand infinite products like this?
For instance, we also have the identity: 
$$ \frac{\zeta^2(s)}{\zeta(2s)} = \sum \frac{2^{v(n)}}{n^s} $$ 
the proof then utilizes infinite product of infinite sums:
$$ \frac{\zeta^2(s)}{\zeta(2s)}  = \prod_p (1 + 2p^{-s} + 2p^{-2s} + \cdots ) = \sum \frac{2^{v(s)}}{n} $$
How do we know the last two equalities coincide?
 A: For the first equality $(*)$ we have the Euler product representation of the zeta function
\begin{eqnarray*}
\zeta(s)= \prod_{p \in \mathbb{P}} (1-p^{-s})^{-1}.
\end{eqnarray*}
From your last comment you seem happy now that
\begin{eqnarray*}
\frac{\zeta(s)}{\zeta(2s)}= \prod_{p \in \mathbb{P}} (1+\frac{1}{p^{s}}).
\end{eqnarray*}
Now consider the terms created by expanding this product
\begin{eqnarray*}
\left(1+\frac{1}{2^{s}}\right)\left(1+\frac{1}{3^{s}}\right)\left(1+\frac{1}{5^{s}}\right) \cdots &=& 1+\frac{1}{2^{s}}+\frac{1}{3^{s}}+\frac{1}{5^{s}}+\frac{1}{6^{s}}+\frac{1}{7^{s}} \cdots \\
&=& \sum_{n=1}^{\infty} \frac{\chi(n)}{n^s} 
\end{eqnarray*}
where $\chi(n)=1$ if $n$ is the product of distinct primes (in other words, $n$ is square-free) and $\chi(n)=0$ otherwise. It is easy to see that $\chi$ is essentially the Mobuis $\mu$ function without the powers of $-1$ to count the parity of the number of distinct primes.
A: This is my attempt to answer my question rigorously, please tell me if it is inappropriate to do so ( I did not want to post edit the OP as it would be too long.) 

Definitions: An infinite product $\prod a_n$ is convergent if exists an index $m$ such that $p_{m,n} = a_m \cdots a_n $ converges to a limit $\hat{p}_m \not=0 $ as $n \rightarrow \infty$.  We define $\prod a_n := a_0 \ldots a_{m-1} \hat{p}_m$. 
An infinite product $\prod b_n$ where $b_n:= 1+ a_n$, is said to be an absolutely convergent infinite product  if  $\prod ( 1 + |a_n| )$ converges. 

Observation: Suppose $\prod a_n$ is conergent. We have $\hat{p}_m $ exists for all $m\in \mathbb{N}$ by definition. Also,  $\lim \hat{p}_m = \frac{\prod a_n}{p_{0,m-1}} \rightarrow 1 $ as $m \rightarrow \infty$ (wlog, product is not equal to $0$). Further $\lim p_m = \lim \hat{p}_m/ \hat{p}_{m+1}  \rightarrow 1 $ as $m \rightarrow \infty$.
Proposition 1: $\prod b_n $, $b_n:=  1 + a_n$, converges iff  $\sum_{n \ge m} \log (1+a_n) $ converges for some $m \in \mathbb{N}$. 
Proof: Suppose $\prod b_n$ converges. From observation $b_n:= 1 + a_n \rightarrow 1$ as $n \rightarrow \infty$ and $\lim \hat{p}_n \rightarrow 1$ as $n \rightarrow \infty$. By continuity, and noting that the argument lies in $\mathbb{C}_{-}$, we have a well defined equation: 
$$ \lim_{n\rightarrow \infty} \log p_{m,n } = \log \hat{p}_n  \Rightarrow \sum_{n \ge m} \log (1+a_n) \text{ converges to some } \alpha $$
Conversely, by taking $e$ to the power of the given result, yields convergene of $\prod b_n$.
Proposition 2: $\prod b_n$, $b_n:=1 + a_n$ converges absolutely iff $\sum |a_n| $ converges. 
Proof: Suppose $\prod b_n$ converges absolutely, then $\prod (1 + |a_n|) $ converges. Hence $\sum \log ( 1 + |a_n| ) $ converges. Now note the inequality, 
$$ \frac{1}{2} |x| \le | \log (1+ x ) | \le 2 |x| $$ 
for $|x| \le \frac{1}{2}$. By applying first proposition, we deduce the result. As a corollary if $\prod b_n$ converges absolutely then $\sum |a_n| $ converges, $\sum \log (1+ a_n) $ converges absolutely, and $\prod b_n$ converges.

Example: In the case of proving identity 
$$ \frac{\zeta^2(s)}{\zeta(2s)} = \sum_{n=1}^\infty \frac{2^{v(n)}}{n^s}, \, ( \sigma > 1 )$$
 we have the infinite product, 
$$ \prod _p (1 + a_p) =  \prod_p  ( 1 + 2p^{-s} + 2 p^{-2s} + \cdots ) $$
Where $a_p = 2( \sum_{n =1 }^\infty  p^{-ns} ) $. Now for some constant $C \in \mathbb{R}_{>0}$ $$\sum_p |a_p| \le \sum_p ( \sum_n \frac{1}{p^{ n \sigma}} )  \le C \sum_p  \frac{1}{p^\sigma} < \infty  $$
Hence, the infinite product converges, and we may apply term by term multiplication, i.e. 
$$ \prod_{p \le P } (1 + a_p) \rightarrow \prod_p (1 + a_p) $$ 
as $P \rightarrow \infty$.  

References: Trench, W. (1999). Conditional Convergence of Infinite Products. The American Mathematical Monthly, 106(7), p.646.
