Questions tagged [estimation-theory]
Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.
1
vote
0answers
16 views
Regressions with two independent variables where one is part of another.
I am doing simple regression analysis where I use one-way fixed effect model to estimate the effects of two variables on dependent variable. The question I am asking is how to interpret these two ...
0
votes
0answers
17 views
Finding the bias of an estimator
Consider the following model:
$$
y_i = a + b x_i + c z_i + w_i,
$$
where $a,b,c$ are unobserved fixed parameters, $x_i$ and $z_i$ are fixed in repeated samples. Assume also $\mathrm{E}[w_i] = 0$ for ...
0
votes
1answer
37 views
showing an estimator is unbiased
If I have the following random variables : $X_1,...,X_n$ with IID distribution of Exp($\lambda$) , $\lambda > 0 $. I want to show two things;
The mean $\overline{X}_n$ is an unbiased estimator of $...
1
vote
0answers
29 views
Stochastic approximation of a Cauchy principle value integral.
Suppose I have a random variable $X\sim f_X(x|\boldsymbol\theta)$ with a well-defined expected value. The usual integral for an analytic solution of this expected value is
$$\operatorname EX=\int_{\...
3
votes
2answers
134 views
Clever Solutions: Find $\min_{a,b} E[(V -(aU+b))^2]$ without differentiation?
Let $(U,V)$ be joint random variables (assume zero means for simplicity). It is well known that
\begin{align}
\min_{a,b} E[(V -(aU+b))^2],
\end{align}
is minimized by $a= \frac{E[VU]}{E[U^2]}$ and $b=...
4
votes
2answers
60 views
Counting coupons in a box
I have a question with regards to counting. I come from a background in quantum computing, so I'm not familiar with the relevant literature here. For the interested reader, I provide below more ...
0
votes
0answers
37 views
Ols estimator with the errors following a bernoulli distribution
I am having trouble understanding how i should approach the following problem:
Given 𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 𝑖 = 1, … , N with 𝜀𝑖 𝑖 = 1,2 … , N being a succession of IID Bernoulli ...
0
votes
0answers
14 views
Finding a dot product using queries.
Alice has two unit vectors, $x, y \in \mathbb{R}^n$. Bob wants to know the value $x\cdot y$. He is allowed to choose any vector $v \in \mathbb{R}^n$, send $v$ to Alice and she will send him $v \cdot x$...
2
votes
1answer
26 views
How can this adjustment of sum be correct?
In our Probability and statistics materials, I run into this equation:
$X_i$ is one result (I hope)
$n$ is a count of the results
$\overline{X_n}$ is sample mean of the results
It is connected ...
0
votes
0answers
41 views
How can I solve this integral equation with the inverse Laplace Transform?
This question is related to Solving an integral equation with inverse Laplace transform.
Let $\alpha,\beta,\mu>0$ with $\alpha/\beta>\mu$ and $X\sim\operatorname{Gamma}(\alpha,\beta)$. I am ...
1
vote
0answers
100 views
What is the unbiased estimator of covariance matrix of N-dimensional random variable?
Suppose $x$ is a random vector in $\mathbb{R}^n$ which is distributed according to $D$.
What is the unbiased estimator of covariance matrix of an N-dimensional random variable?
When $y$ is a i.i.d. ...
0
votes
0answers
15 views
Sample moments Intuitive explanation
Can anyone intuitively explain to me the definition I have posted above? Also, what is the purpose of this definition i.e. how do you use it?
0
votes
0answers
13 views
Nonlinear Gaussian State Space with Linear Observation Matrix Derivation
For the following State Space Model:
$x_k = f(x_{k-1}) + v_k ~~ \sim ~ \Bbb N(0_{n_{v \times 1}}, \sum_v) \\
y_k = Cx_{k} + w_k ~~ \sim ~ \Bbb N(0_{n_{w \times 1}}, \sum_w)$
where $f: \Bbb R^{n_{...
0
votes
0answers
23 views
Constructing an upper confidence limit for $σ^2$
Suppose that $X_1, ..., X_n$ form a random sample from the normal distribution with unknown mean µ and
unknown standard deviation σ.
Construct an upper confidence limit U(X1, ..., Xn) for $σ^2$ such ...
0
votes
1answer
30 views
Showing that an estimator is consistent
Let $X_1,X_2,\ldots,X_n$ be a random sample from $\mathcal{N}(\theta,1)$. Consider the following (randomized) estimator of $\theta$ given a sample of size $n$:
$$
\hat{\theta}_n = \bar{X} + \begin{...
0
votes
0answers
19 views
Showing that a estimator is consistent
Let $X_1,X_2,\ldots,X_n$ be a random sample from $\mathcal{N}(\theta,1)$. Consider the following (randomized) estimator of $\theta$ given a sample of size $n$:
$$
\hat{\theta}_n = \bar{X} + \begin{...
2
votes
1answer
39 views
Which loss function does the maximum likelihood estimator minimize?
I'm trying to understand Maximum Likelihood estimators in the context of general estimation theory. I know Bayesian estimator minimizes mean squared loss, MAP estimator minimizes all-or-nothing loss (...
0
votes
0answers
25 views
Finding method of moments estimators for uniform distribution.
I am trying to do the following question:
I was following Michael Hardy's answer, and I got $\hat{b}=2\bar{x}-\hat{a}$ and the equation $\hat{a}^{2}-2\bar{x}\hat{a}-\frac{3\sum (x_{i}-\bar{x})^{2}}{n}...
0
votes
2answers
66 views
Finding expected value using integration by parts.
Can someone check whether my solutions for $E(X)$ is okay?
For (a), I found $E(X) = \frac{\theta +1}{\theta}$ and for (b), I applied integration by parts twice to get $2\theta ^{4}$. I feel as though ...
1
vote
0answers
48 views
Is there a more efficient expected value estimator than the sample average?
I'm wondering if there is a known estimator for expected value that is more efficient than the sample average.
If that is not the case for an arbitrary random variable, then maybe there are examples ...
0
votes
1answer
35 views
Unbiased estimator having a deviation less than $0.05$
In a mass production of items produced indepedently of eachother the probability of an item being defect is $p$. An unbiased estimator for $p$ is
$\hat p = \frac{X}{n}$ where $X$ = amount of items ...
2
votes
0answers
71 views
Laplace Method - Estimation of an integral
I am just working on a paper by Shinzo Watanabe on "Asymptotic Evaluations of Wiener Functional expactations":
$μ(dx)$ is the $d$-dimensional Gaussian distributions
So that is the interesting ...
0
votes
0answers
32 views
Sequential Hypothesis Test
Consider a random variable Y and the pair of simple hypotheses:
$H_0 : Y \sim N(0,10)$
$H_1 : Y \sim N(m,10)$
Here $m$ is an unknown mean which is sampled from a distribution $N(0,1)$ and then ...
2
votes
2answers
40 views
generalized concept of “expected value” for outcome sets without a vector space structure
Does the following concept has a name ?
Starting point : "expected value"
What we call "expected value" for a random variable $X$ requires a vector space structure on its outcome set. Say for some ...
0
votes
1answer
103 views
Calculating a bias given the MOM and MLE
Question: A company manufactured objects, starting from 1 to N. One of the objects is selected, and serial number is 888. We are asked to find things like the MOM, MLE etc. which I managed without ...
1
vote
0answers
33 views
Estimate parameters of Wishart matrix.
Given a sequence of real Wishart matrices $W_1 , \cdots , W_k \sim \mathcal{W}_m(n,\Sigma)$ where $\Sigma$ is a singular matrix. Are there good estimates for the degrees of freedom?
The MLE for $\...
0
votes
0answers
71 views
Properties of conditional expectation-tower property
I was reading Prof. Amir Dembo's lecture notes on Stochastic Process. There are two exercises that I cannot figure out.
The first exercise is from 2.3.7 of the notes:
Let $Var(Y|\mathcal{G})= \...
0
votes
0answers
146 views
Exponential family distribution and sufficient statistic.
The exponential distribution family is defined by pdf of the form:
$$ f_x=(x;\theta) = c(\theta) g(x) exp \Big[\sum_{j=1}^l G_j(\theta) T_j(x)]$$
Where $\theta \in \Theta$ and $c(\theta)>0$ And $...
0
votes
1answer
27 views
Exmaple on convergence of random variables - change of integral boundaries in normal distribution
Suppose X$_n$ folowes $N(\mu,\frac{1}{n^2})$. Thus
$$ F_{X_n} = \frac {n}{\sqrt{2\pi}}\int_{-\infty}^xexp(-\frac{1}{2}n^2y^2)dy= \frac {n}{\sqrt{2\pi}}\int_{-\infty}^{nx} exp(-\frac{1}{2}z^2)dz$$.
...
0
votes
0answers
36 views
Variance of Linear MMSE estimator from three measurements.
I am here presenting a question on calculating the variance of an Estimator. This is a problem in localization of vehicle using ranging from Vehicular Ad hoc Network and Change in location from ...
1
vote
0answers
31 views
Writing the Extended Kalman Filter
Suppose that you have a system:
\begin{cases}\dot{x}(t)=A(u(t))x(t)+d_1(t)\\y(t)=f(x(t))+d_2(t),\end{cases}
where $A:\mathbb{R}\longrightarrow\mathbb{R}^{n\times n}$ is a matrix function, $u:\mathbb{R}...
0
votes
1answer
30 views
Estimate how many tracks the city has.(Almost done, but can not find whether estimator is biased or not)
I try to estimate the number of tracks in the city by observing their serial numbers. Assume that the serial numbers are drawn from a uniform probability density ranging from 0 to an unknown parameter ...
0
votes
1answer
633 views
Beta distribution as a member of the exponential family
I come across the beta distribution quite frequently when solving exercises for my statistics class. However, I have not been able to fully grasp how to work with it.
Exponential family form is:
$$...
3
votes
1answer
188 views
Gamma distribution family and sufficient statistic
Let X $=X_1,...X_n$ be a random sample iid from the probability density function:
$$ f(x;\theta)=\frac{\Gamma(\theta)\sin(\pi\theta)\theta^{1-\theta}}{\pi}e^{-\theta x}x^{-\theta}$$ $x>0, 0<\...
1
vote
2answers
30 views
ML estimation with given samples
Let $X_i,...,X_n$ be a random independent sample from a distribution with pdf
$$ f(x;\theta)= (\theta + 1)x^{-(\theta+2)},$$
where $x>0$, and $\theta > 0$. What is the ML estimate for the ...
0
votes
1answer
116 views
derive sufficient statistic from a random independent sample from a weibull distribution
Suppose $X_i$ is a random independent sample from a Weibull distribution
$$ f(x) = \frac{\beta}{\theta^\beta}x^{\beta-1}\exp\big(-(\frac{x}{\theta})^\beta\big)$$
Find a sufficient statistic for ...
0
votes
1answer
69 views
derive asymptotic distribution of the ML estimator
Let $x$ be a random variable with probability density (pdf)
$$f(x)= (\theta +1)x^\theta $$
where $\theta >-1$.
The expressions for its mean and variance are
$$E(X)= \frac{\theta + 1}{\theta +...
0
votes
1answer
126 views
unbiased pool estimator of variance
I'm not sure I'm calculating the unbiased pooled estimator for the variance correctly.
Assuming 2 samples where $\sigma_1 = \sigma_2 = \sigma$ and is uknown, these are my definitions:
Sample ...
0
votes
0answers
23 views
need help identifying a formula for “pseudo-entropy”
Maintaining some old code, I've come across:
$$\text{pseudo-entropy} = -x \log(x) + x ^{0.45} \cdot (1 - x) ^ {16}$$
I simply need a name for this formula so I can read up on what it's supposed to ...
0
votes
0answers
16 views
Bias in wilson estimator
In case a population is small when estimating proportion p is small and the samle size n is small, with the typical estimator for population proportion $\hat{p_1}=\frac{X}{n}$ one might easily get ...
1
vote
2answers
90 views
Show that $\frac1n(\sum_{i=1}^n\log\frac{1}{1-X_i})^3$ is a sufficient statistic for $\beta$ in a Beta$(\alpha,\beta)$ density
Suppose that $X_1,...,X_n$ From a random sample from a distributio with density $$ f(x) = \frac{\Gamma{(\alpha+\beta)}}{\Gamma{(\alpha)}\Gamma{(\beta)}}x^{\alpha-1}(1-x)^{\beta-1} \quad,\text{ if }\...
0
votes
0answers
62 views
Derive the asymptotic distribution of maximum likelihood estimator
Suppose that $X_1,...,X_n$ is a random sample from a distribution with pdf :
$$ f(x,\theta)= \frac{\theta^3}{2}x^2e^{-\theta x} \space for \space 0,x,\infty$$
I found that $l(\theta)=-nlog(2)+3nlog(\...
0
votes
1answer
330 views
Calculating variance of an estimator
Given that $ \operatorname{Var}(x)=\frac{3}{4}\theta^2$, I want t find the variance of estimator $\hat{\theta_1} = \frac{2n}{3}\sum_{i=1}^nX_i$. EDIT: $X_1,...,X_n$ are independent and identically ...
1
vote
1answer
52 views
Find a sufficient statistic. [closed]
Suppose that $X_1,\ldots,X_n$ is a random sample from a distribution with pdf
$$ f(x;\theta)=\frac{\theta^3}{2}x^2e^{-\theta x}, \quad 0<x<\infty$$
where $0<\theta<\infty$
Find a ...
0
votes
0answers
119 views
calculate variance of unbiased estimator in Rayleigh distribution
Given : $\hat{\theta} = \frac{\sum X_i^2}{2n} $ and $ E(X^2) = 2\theta $. $\hat{\theta}$ is unbiased estimator for $\theta$ based on i.i.d sample from $f(x;\theta)= \frac{x}{\theta}e^{\frac{-x^2}{2\...
0
votes
0answers
16 views
Induced bias in r.v. A $= \pi R^2$ where R is unbiased measurement of fixed unknown.
I have particularly poor and incomplete notes that refer to induced bias in the random variable A of the area of a circle that is dependent on random variable R which is the measured radius (true ...
1
vote
0answers
69 views
Variance of mean estimator with variable sample size
I'm looking at a random variable that takes vectors $\newcommand{\vv}{\mathbf{v}} \vv_1, \dotsc, \vv_n \in \mathbb{R}^d$ and calculates their average, after applying "blankout" noise to them. So we ...
2
votes
1answer
33 views
Estimation $\mu^2$ under certain conditions.
Let $X_1,X_2,....,X_n$ be a random sample of size $n$ from a population with cdf $F()$. Let $E(X)=\mu$ exist. Then estimate $\mu^2$ unbiasedly for the following three cases:-
(i) $Var(X)=\sigma^2$ ...
2
votes
1answer
41 views
Estimator for $\theta$ using the method of moments
Exercise :
Using the method of moments, find the estimator for $\theta$ for a random sample $X_1, \dots, X_n$ that follows the distribution with pdf $f(x) = \theta x^{-2}, \; 0 < \theta \leq x &...
0
votes
0answers
10 views
Particle Filters: whether current measurement depends on previous measurement
Recently I was learning basic estimation theory. I am confused that whether the measurement $z_k$ is related to previous measurements $z_{1:k-1}$.
A quick recap of the estimation problem.
Given ...
