Variance of squared norm of multivariate normal vector Suppose, that we have a secuence of independent random variables $\{w_i\}_{i=0}^n$ from $\mathcal{N}(0,\mathbb{I}_d)$. We consider the following weighted sum defined for some $\beta \in (0,1)$
$$
\theta = \sum_{i=0}^n (1 - \beta)^i w_i
$$
We are interested in the value of $\text{Var}(\|\theta\|_2^2)$. 
My approach.
I have started with calculation of $\mathbb{E}(\|\theta\|_2^2)$. It appears to be $\frac{d}{\beta(2-\beta)}$. Next, I moved to estimation of
$$
\mathbb{E}(\|\theta\|_2^4) 
= 
\mathbb{E}\left(\| \sum_{i=0}^n (1 - \beta)^i w_i \|_2^4\right)
= 
\sum_{i=0}^n (1 - \beta)^i \mathbb{E}(\|w_i\|_2^4)
+
\sum_{i,j=0, i\ne j}^n 3(1 - \beta)^{2i+2j} \mathbb{E}(\|w_i\|^2)\mathbb{E}(\|w_i\|^2)
$$
Then I have found that $\mathbb{E}(\|w_i\|_2^4) = d^2  + 2d$ and $\mathbb{E}(\|w_i\|^2) = d$. Then I used the following equality
$$
\sum_{i,j=0}^n (1 - \beta)^{2i+2j} \approx \frac{1}{(1-(1-\beta))^2}.
$$
to obtain for small $\beta$
$$
\mathbb{E}(\|\theta\|_2^4) 
\approx 
\sum_{i,j=0, i\ne j}^n 3(1 - \beta)^{2i+2j} d^2
\approx
\sum_{i,j=0}^n \frac{3d^2}{(1-(1-\beta))^2}.
$$
Then
$$
\text{Var}(\|\theta\|_2^2) = \mathbb{E}(\|\theta\|_2^4) - (\mathbb{E}(\|\theta\|_2^2))^2 = \frac{2d^2}{(1-(1-\beta))^2}.
$$
But computations show that the variance is 
$$
\text{Var}(\|\theta\|_2^2) = \frac{2d}{(1-(1-\beta))^2},
$$
which is $d$ times smaller than mine. 
 A: Firstly, I'm assuming you're interested in the limit as $n\to\infty$ since $n$ does not appear in your expressions. (One can show the a.s. limit exists by either Levy's theorem or Kolmogorov's one-series theorem.) I will write $\theta_n$ for the expression you have written and $\theta$ for its limit. I will also use the characteristic function, which avoids the need to justify results such as $\mathbb E(\|\theta_n\|_2^2)\to\mathbb E(\|\theta\|_2^2)$.
The characteristic function of $\theta_n$ is
$$\phi_n(t)=\mathbb E(e^{i\langle t,\theta_n\rangle})=\prod_{j=0}^n\mathbb E(e^{i\langle t,(1-\beta)^j\omega_j\rangle})=\prod_{j=0}^n\mathbb e^{-(1-\beta)^{2j}\|t\|_2^2/2}=\exp\left(-\frac12\sum_{j=0}^n(1-\beta)^{2j}\|t\|_2^2\right)\\
=\exp\left(-\frac{1-(1-\beta)^{2(n+1)}}{2(1-(1-\beta)^2)}\|t\|_2^2\right)\to\exp\left(-\frac1{2\beta(2-\beta)}\|t\|_2^2\right).$$
This shows that $\theta$ is a (centered) Gaussian with covariance matrix $\Sigma=\frac1{\beta(2-\beta)}\mathbb I_d$. This makes the calculations very straightforward: we have
$$\mathbb E(\|\theta\|_2^2)=\frac{d}{\beta(2-\beta)}$$
and
$$\mathbb E(\|\theta\|_2^4)=\frac{d}{\beta^2(2-\beta)^2}+\frac{3d(d-1)}{\beta^2(2-\beta)^2}.$$
This implies
$$\operatorname{Var}(\|\theta\|_2^2)=\frac{2d(d-1)}{\beta^2(2-\beta)^2}$$
which is fairly close to what you have found.
I cannot think of why the computations would be wrong in this case. Perhaps if you provided some more context to this problem I could see where the issue is coming from.
