The procedure for finding homology and cohomology of the spaces in question is a neat little trick. From here on out, I'll just treat the homology case, but the cohomology follows from the same arguments. Collapse the $S^{n-1}$ you're gluing along to a point- this turns $M\# N$ into $M\vee N$. Since $(M\# N, S^{n-1})$ is a good pair, the homology can be identified with the relative homology of the pair $(M\# N,S^{n-1})$. From this, we get the following long exact sequence:
$$\cdots\to \widetilde{H_i}(S^{n-1})\to \widetilde{H_i}(M\# N) \to \widetilde{H_i}(M\vee N)\to\cdots$$
By a simple Mayer-Vietoris argument, we have that $\widetilde{H}_i(M\vee N)\cong \widetilde{H}_i(M)\oplus \widetilde{H}_i(N)$. Since $\widetilde{H_i}(S^{n-1})$ is zero except for $i=n-1$, we have automatically that $H_i(M\# N)\cong H_i(M\vee N)\cong H_i(M)\oplus H_i(N)$ for $i\neq n-1,n$. The only interesting case is as follows:
$$0\to \widetilde{H_n}(M\# N)\to \widetilde{H_n}(M\vee N) \to \widetilde{H}_{n-1}(S^{n-1})\to \widetilde{H}_{n-1}(M\# N)\to \widetilde{H}_{n-1}(M\vee N) \to 0$$
Now, we start getting into some casework depending on whether none, one, or both of $M,N$ are orientable. In the case that both are orientable, the above sequence turns into
$$0\to \mathbb{Z} \to \mathbb{Z}\oplus\mathbb{Z} \to \mathbb{Z} \to \widetilde{H}_{n-1}(M\# N)\to \widetilde{H}_{n-1}(M\vee N) \to 0$$
as their connected sum is also orientable. From this, we see that $\widetilde{H}_{n-1}(M\# N)\to \widetilde{H}_{n-1}(M\vee N)$ must be an isomorphism.
If just one of $M,N$ is orientable, then their connected sum is non-orientable, and the following happens:
$$0\to 0 \to \mathbb{Z}\oplus0 \to \mathbb{Z} \to \widetilde{H}_{n-1}(M\# N)\to \widetilde{H}_{n-1}(M\vee N) \to 0$$
in which case we still have that that $\widetilde{H}_{n-1}(M\# N)\to \widetilde{H}_{n-1}(M\vee N)$ must be an isomorphism.
If neither of $M,N$ are orientable, then their connected sum is non-orientable, in which case the long exact sequence does the following:
$$0\to 0 \to 0 \to \mathbb{Z} \to \widetilde{H}_{n-1}(M\# N)\to \widetilde{H}_{n-1}(M\vee N) \to 0$$
and thus $\widetilde{H}_{n-1}(M\# N)$ is an extension of $\widetilde{H}_{n-1}(M\vee N)$ by $\mathbb{Z}$. To figure out what extension it is, one needs to inspect the map $S^{n-1}\to M\# N$ and the corresponding map on homology. Nothing too surprising can happen- $H^{n-1}(M\# N)$ is the direct sum of a free abelian group and a finite abelian group.
Note that during this argument, it was never necessary to talk about the orientation of the gluing- so $M\# N$ and $M\#_{rev}N$ have the same homology/cohomology. No description of $S^{n-1}$ was ever used except for it having reduced homology only in degree $n-1$, so the process does not care very much about what dimension your manifolds are.
Now, for the example where $M=S^1\times S^3$ and $N=\mathbb{C}P^2$. $M$ has homology $H_0\cong H_1\cong H_3\cong H_4\cong \mathbb{Z}$ and all other groups zero, while $N$ has homology $H_0\cong H_2\cong H_4\cong \mathbb{Z}$ and all other groups zero. Using the procedure above, we have that the homology of $M\# N$ is as follows: $H_0\cong H_1\cong H_2\cong H_3\cong H_4\cong \mathbb{Z}$. The result is the same for $M\#_{rev}N$.
Addendum: It has been pointed out in the comments that the arguments about what happens with the maps between the various copies of $\Bbb Z$ are not quite complete. The key gap is showing that the map from $\widetilde{H}_n(M\vee N)\to \widetilde{H}_{n-1}(S^{n-1})$ is surjective when at least one of $M,N$ are orientable. The fix is reasonable, and is presented below to make this answer as complete as possible.
The sequence for a good pair gives that the map $H_n(M\# N,S^{n-1})\to H_{n-1}(S^{n-1})$ is given by taking the boundary of relative chains. Without loss of generality, $M$ is orientable, so consider the relative class in $H_n(M\# N,S^{n-1})$ which is the fundamental class of $M$ minus the disc we delete to perform the connected sum operation. This has boundary exactly the fundamental class of $S^{n-1}$ (up to possibly a sign change for orientation issues) by the definition of how we chose $S^{n-1}$ to glue along. As we identified $H_n(M\# N,S^{n-1}) \cong \widetilde{H}_n(M\vee N) \cong \widetilde{H}_n(M)\oplus \widetilde{H}_n(N)$, we see that both the map $\Bbb Z\oplus\Bbb Z\to \Bbb Z$ and $\Bbb Z\oplus 0\to \Bbb Z$ are surjective. Thus the conclusion about $\widetilde{H}_{n-1}(M\# N)\to \widetilde{H}_{n-1}(M\vee N)$ being an isomorphism when at least one of $M,N$ is orientable holds.