Expectation, Variance and Moment estimator of Beta Distribution

Question

I'm given a beta distributed random variable: $X \sim \text{Beta}_{(\theta, 1)} =: \mathbb{P}_\theta$. Where $\theta \geq q$ and

$$\mathbb{f}_\theta(x) = \theta \cdot x^{\theta-1} \cdot \mathbb{1}_{[0,1]}(x)$$

I was asked to compute the expectation and variance of $X$ and came up with the following solution:

$$\mathbb{E}X = \int_\mathbb{R} x \cdot \mathbb{f}_\theta(x) \, \text{dx} = \theta \int_0^{1} x^{\theta} \, \text{dx} = \frac{\theta}{\theta + 1}$$

$$\mathbb{E}X^2 = \int_\mathbb{R} x^2 \cdot \mathbb{f}_\theta(x) \, \text{dx} = \theta \int_0^{1} x^{\theta+1} \, \text{dx} = \frac{\theta}{\theta + 2}$$

$$\text{Var}X = \mathbb{E}X^2 - (\mathbb{E}X)^2 = \frac{\theta}{(\theta+1)^2(\theta+2)}$$

Are these computations correct or did I end up making a mistake? Now let $X_1, \dots, X_n \sim \text{Beta}_{(\theta,1)}$ iid. How can I justify that the moment estimator of $\theta$ is given by $\hat\theta:n = \frac{\bar X_n}{1 - \bar X_n}$? Is this estimator consistent?

I'd say that the estimator is consistent. The Expectation and Variance of $X_1$ is finite. Now after the strong law of large numbers we get

$$\lim_{n \rightarrow \infty} \hat\theta_n = \frac{\mathbb{E}X_1}{1 - \mathbb{E}X_1} = \theta$$

Therefore the estimator should be consistent, but how can I show that the moment estimator of $\theta$ is given by $\hat\theta_n$? I'm familiar with the method of moments, but don't understand how to apply it in this case.

Answer 1

The idea behind the Method of Moments estimator is the following.

Let suppose we have $\underline{X}$ $=$ $(X_{1},...,X_{n})$ iid observations, distributed according to $f(\cdot \mid \boldsymbol{\theta})$, $\boldsymbol{\theta}$ $\in$ $\Theta$ $\subseteq$ $\mathbb{R}^{d}$.

Define:

$$\mu_{k} = E(X_{i}^{k}) = \mu_{k}(\theta_{1},...,\theta_{d})$$

the $k$-th moment of $X_{i}$ and

$$m_{k} = \frac{1}{n}\sum_{i = 1}^{n}X_{i}^{k}$$

the $k$-th sample moment. Then we estimate the vector of parameters $\boldsymbol{\theta}$ = $(\theta_{1},...,\theta_{d})$ solving the following system of equations:

\begin{cases} m_{1} = \mu_{1}(\theta_{1},...,\theta_{d})\\ .\\ .\\ m_{d} = \mu_{d}(\theta_{1},...,\theta_{d})\end{cases}

leading to $(\hat{\theta_{1}},...,\hat{\theta_{d}})$ $=$ $\boldsymbol{\hat{\theta}}$, our method of moments estimator.

Now, consider a generic beta distribution:

$$f(x_{i} \mid \theta, \alpha) = \frac{\Gamma(\theta + \alpha)}{\Gamma(\theta)\Gamma(\alpha)}x_{i}^{\theta - 1}(1 - x_{i})^{\alpha - 1}I(0 < x_{i} < 1)$$

In this case, $\boldsymbol{\theta}$ $=$ $(\theta, \alpha)$, hence we need to consider the first two moments.

The first moment of $X_{i}$ is:

$$E(X_{i}) = \frac{\theta}{\theta + \alpha}$$

while the second moment can be proved to be:

$$E(X_{i}^{2}) = \frac{\theta(\theta + 1)}{(\theta + \alpha)(\theta + \alpha + 1)}$$

Now, at this point, for the first moment in the sample, we have:

$$m_{1} = \frac{1}{n}\sum_{i = 1}^{n}X_{i} = \overline{X_{n}}$$

while for the second moment in the sample we exploit the sample variance:

$$s^{2} = m_{2} - m_{1}^{2} \Rightarrow m_{2} = s^{2} + m_{1}^{2}$$

and, at this point, we apply the system of equations described before to this case, to be then solved for both $\theta$ and $\alpha$:

\begin{cases} m_{1} = \overline{X_{n}} = E(X_{i}) = \frac{\theta}{\theta + \alpha}\\ m_{2} = s^{2} + m_{1}^{2} = E(X_{i}^{2}) = \frac{\theta(\theta + 1)}{(\theta + \alpha)(\theta + \alpha + 1)}\end{cases}

which would yield the two Method of Moments estimators $\hat{\theta}$ and $\hat{\alpha}$, and this would be in the general case with both parameters of the beta unknown. In your case, the problem is simplified being $\alpha$ $=$ $1$ known, so that we have to estimate only $\theta$ using the first sample moment, that is we have only the first equation to solve with $\alpha$ $=$ $1$:

$$\overline{X_{n}} = \frac{\theta}{\theta + 1} \Rightarrow \hat{\theta} = \frac{\overline{X_{n}}}{1 - \overline{X_{n}}}$$

Hope it clarifies.