Wikipedia gives $$\sum_{k=2}^\infty(\zeta(k)-1)=1,\quad\sum_{k=1}^\infty(\zeta(2k)-1)=\frac34,\quad\sum_{k=1}^\infty(\zeta(4k)-1)=\frac78-\frac\pi4\left(\frac{e^{2\pi}+1}{e^{2\pi}-1}\right)$$ from which we can easily find $\sum_{k=1}^\infty(\zeta(2k+1)-1)$ and $\sum_{k=1}^\infty(\zeta(4k+2)-1)$. From here it's natural to ask the following
Question: Is there a known closed form for $\sum_{k=1}^\infty(\zeta(4k+1)-1)$?
Related: Closed form for $\sum\limits_{k=1}^{\infty}\zeta(4k-2)-\zeta(4k)$ and Closed form for $\sum_{k=1}^{\infty} \zeta(2k)-\zeta(2k+1)$.
Progress
Note that we are done once we have a closed form for $\sum_{k=1}^\infty(\zeta(4k+1)-\zeta(4k+3))$. So I tried the same approach as in one of the questions above and this is what I got:
We have $$\zeta(4k+1)-\zeta(4k+3)=\sum_{n\geq2}^\infty\left(1-\frac1{n^2}\right)\frac1{n^{4k+1}}$$
Hence
$$\sum_{k\geq1}\zeta(4k+1)-\zeta(4k+3)=\sum_{n\geq2}\left(1-\frac1{n^2}\right)\sum_{k\geq1}\frac1{n^{4k+1}}=\sum_{n\geq2}\left(1-\frac1{n^2}\right)\frac1{n^5}\frac1{1-\frac1{n^4}}\\=\sum_{n\geq2}\frac1{n^3+n^5}.$$
In the same way I get $$\sum_{k\geq1}\zeta(4k+1)-\zeta(4k)=-\sum_{n\geq2}\frac1{n+n^2+n^3+n^4}$$ and $$\sum_{k\geq1}\zeta(4k+1)-\zeta(4k+2)=\sum_{n\geq2}\frac1{n^2+n^3+n^4+n^5}.$$ It suffices (in fact it is equally hard) to evaluate any of these series.