Cramer-Rao bound for LS estimator It's a problem from Machine Learning: A Bayesian And Optimization Perspective (problem 3.7):
Derive the Cramer-Rao bound for the LS estimator, when the training data result from the linear model $$y_n=\theta x_n+\eta_n, n=1, 2, ..., N$$ where $x_n$ and $\eta_n$ are i.i.d sample of a zero mean random variable, with variance of $\sigma^2_x$ and a Gaussian one with zero mean and variance of $\sigma^2_{\eta}$, respectively. Assume, also, that x and η are independent. Then show that the LS estimator $$\theta=\frac{\sum^N_{n=1}{x_n y_n}}{\sum^N_{n=1}{x_n^2}}$$ achieves the CR bound only asymptotically.
It needs the pdf of $y_n$ which is the sum of two independent random variables, am I supposed to use the convolution formula (the theorem of summation of two random variables)? It seems the calculation would go extremely difficult because of the integration：$$\text{y}\sim\int_{\mathbb{R}}{\frac{1}{\theta}p(\frac{y}{\theta})\frac{1}{\sqrt{2\pi \sigma_{\eta}^2}}\text{exp}(-\frac{(y-u)^2}{2\sigma_{\eta^2}})\text{du}}, \text{where}\,\,p\,\,\text{is the pdf of x}$$ there is a $\theta$ in $p$ which mostly causes the computational difficulty. 
Any helps or hints will be appriciated. 
 A: Edit: Included computation of the cramer-Rao-bound
Note that $Y|X \sim  \mathcal{N}(\theta X;\sigma_\eta^2)$
We have that $I_{(X,Y)}(\theta)  = I_{Y|X}(\theta) = \mathbb{E}_{\theta,x} [ \frac{\partial \ln p(y|x)}{\partial \theta}^2] = \mathbb{E}_{\theta,x}[\frac{X^2}{\sigma_\eta^2}] = \frac{\sigma_x^2}{\sigma_\eta^2} $
By the "chain rule" for fisher information.
Also, for the asymptotics you will not need to evaluate the density if you do it in the following way:
First, separate the terms:
$\frac{\sum_n x_n y_n}{\sum_n x_n^2} = \theta \frac{\sum_n x_n^2}{\sum_n x_n^2} + \frac{\sum_n x_n \eta_n}{\sum_n x_n^2} = \theta + \frac{\sum_n x_n \eta_n}{\sum_n x_n^2} $
Now check the asymptotics:
$ \sqrt{n}(\theta + \frac{\sum_n x_n \eta_n}{\sum_n x_n^2} - \theta) = \sqrt{n} ( \frac{\sum_n x_n \eta_n}{\sum_n x_n^2}) = \sqrt{n} ( \frac{\frac{\sum_n x_n \eta_n}{n}}{\frac{\sum_n x_n^2}{n}})  = \sqrt{n} \frac{\overline{X \eta}}{\overline{X^2}}$
Now, by the law of large numbers, 
$ \overline{X^2} \overset{P}{\rightarrow} \sigma_x^2 $
And, by the central limit theorem (and the independence of $X$,$\eta$):
$\sqrt{n} \overline{X \eta} \overset{D}{\rightarrow}  \mathcal{N}(0,\sigma_x^2 \sigma_\eta^2)$
Hence, by the Slutsky theorem/continuos mapping theorem, you get that:
$ \sqrt{n} \frac{\overline{X \eta}}{\overline{X^2}} \overset{D}{\rightarrow} \mathcal{N}(0,\frac{\sigma_\eta^2}{\sigma_x^2})$
which is the inverse of the Fisher information, as required.
A: This is partly a summary, and thanks to works by @a_student, @StubbornAtom, and @jld in https://stats.stackexchange.com/q/320600. This may be the final answer to this problem, please point out mistakes if you find some.
First, we compute the C-R bound of the estimator, by the definition of 
$$
I_{(X, Y)}(\theta)=-\mathbb{E}_{\left( X,Y \right)}\text{[}\frac{\partial ^2}{\partial \theta ^2}\ln p\left( X,Y;\theta \right) \text{]}, \text{where}\,\,X=\left( x_1,x_2,...,x_N \right) ^T, Y=\left( y_1,y_2,...,y_N \right) ^T
$$
and the chain rule, we have that
$$
I_{\left( X,Y \right)}\left( \theta \right) 
=-\mathbb{E}_{(X, Y)}\left[ \frac{\partial ^2}{\partial \theta ^2}\ln p\left( Y|X;\theta \right) \right] +0
$$
as $X$'s distribution doesn't consists of $\theta$, and we have $Y|X\sim \mathcal{N}\left( \theta X, \sigma _{\eta}^{2}I \right) $, after computation we get
$$
C-R\,\,bound=\frac{1}{I\left( \theta \right)}=\frac{\sigma _{\eta}^{2}}{N\sigma _{x}^{2}}
$$
Then, compute the variance of the estimator $\hat{\theta}$. By the computation in @jld 's work we have 
$$
Var(\hat{\theta})=σ^2_\eta\,\mathbb{E}_X[\frac{1}{X^TX}]
$$
we don't have a Gaussion X, but it suffices to prove that $\mathbb{E}_X[\frac{1}{X^TX}]\rightarrow 0$, as $N\rightarrow\infty$. 
By the law of large numbers, $\overline{X^T X}\xrightarrow{P}\sigma^2_x$, then by continuous mapping theorem we have
$$
\frac{1}{X^TX}=\frac{1}{N}\frac{1}{\overline{X^TX}}\xrightarrow{P}0
$$
we finally get the result that $\hat{\theta}$ asymptotically attains the bound, since both of them tends to 0 as $N\rightarrow\infty$.
