Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [estimation-theory]

Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

0
votes
0answers
17 views

Ols estimator with the errors following a bernoulli distribution

I am having trouble understanding how i should approach the following problem: Given 𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 𝑖 = 1, … , N with 𝜀𝑖 𝑖 = 1,2 … , N being a succession of IID Bernoulli ...
0
votes
0answers
13 views

Finding a dot product using queries.

Alice has two unit vectors, $x, y \in \mathbb{R}^n$. Bob wants to know the value $x\cdot y$. He is allowed to choose any vector $v \in \mathbb{R}^n$, send $v$ to Alice and she will send him $v \cdot x$...
2
votes
1answer
22 views

How can this adjustment of sum be correct?

In our Probability and statistics materials, I run into this equation: $X_i$ is one result (I hope) $n$ is a count of the results $\overline{X_n}$ is sample mean of the results It is connected ...
0
votes
0answers
32 views

How can I solve this integral equation with the inverse Laplace Transform?

This question is related to Solving an integral equation with inverse Laplace transform. Let $\alpha,\beta,\mu>0$ with $\alpha/\beta>\mu$ and $X\sim\operatorname{Gamma}(\alpha,\beta)$. I am ...
1
vote
0answers
34 views

What is the unbiased estimator of covariance matrix of N-dimensional random variable?

Suppose $x$ is a random vector in $\mathbb{R}^n$ which is distributed according to $D$. What is the unbiased estimator of covariance matrix of an N-dimensional random variable? When $y$ is a i.i.d. ...
0
votes
0answers
14 views

Sample moments Intuitive explanation

Can anyone intuitively explain to me the definition I have posted above? Also, what is the purpose of this definition i.e. how do you use it?
0
votes
0answers
13 views

Nonlinear Gaussian State Space with Linear Observation Matrix Derivation

For the following State Space Model: $x_k = f(x_{k-1}) + v_k ~~ \sim ~ \Bbb N(0_{n_{v \times 1}}, \sum_v) \\ y_k = Cx_{k} + w_k ~~ \sim ~ \Bbb N(0_{n_{w \times 1}}, \sum_w)$ where $f: \Bbb R^{n_{...
0
votes
0answers
21 views

Constructing an upper confidence limit for $σ^2$

Suppose that $X_1, ..., X_n$ form a random sample from the normal distribution with unknown mean µ and unknown standard deviation σ. Construct an upper confidence limit U(X1, ..., Xn) for $σ^2$ such ...
0
votes
1answer
22 views

Showing that an estimator is consistent

Let $X_1,X_2,\ldots,X_n$ be a random sample from $\mathcal{N}(\theta,1)$. Consider the following (randomized) estimator of $\theta$ given a sample of size $n$: $$ \hat{\theta}_n = \bar{X} + \begin{...
0
votes
0answers
16 views

Showing that a estimator is consistent

Let $X_1,X_2,\ldots,X_n$ be a random sample from $\mathcal{N}(\theta,1)$. Consider the following (randomized) estimator of $\theta$ given a sample of size $n$: $$ \hat{\theta}_n = \bar{X} + \begin{...
2
votes
0answers
18 views

Which loss function does the maximum likelihood estimator minimize?

I'm trying to understand Maximum Likelihood estimators in the context of general estimation theory. I know Bayesian estimator minimizes mean squared loss, MAP estimator minimizes all-or-nothing loss (...
0
votes
0answers
18 views

Finding method of moments estimators for uniform distribution.

I am trying to do the following question: I was following Michael Hardy's answer, and I got $\hat{b}=2\bar{x}-\hat{a}$ and the equation $\hat{a}^{2}-2\bar{x}\hat{a}-\frac{3\sum (x_{i}-\bar{x})^{2}}{n}...
0
votes
2answers
44 views

Finding expected value using integration by parts.

Can someone check whether my solutions for $E(X)$ is okay? For (a), I found $E(X) = \frac{\theta +1}{\theta}$ and for (b), I applied integration by parts twice to get $2\theta ^{4}$. I feel as though ...
1
vote
0answers
44 views

Is there a more efficient expected value estimator than the sample average?

I'm wondering if there is a known estimator for expected value that is more efficient than the sample average. If that is not the case for an arbitrary random variable, then maybe there are examples ...
0
votes
1answer
33 views

Unbiased estimator having a deviation less than $0.05$

In a mass production of items produced indepedently of eachother the probability of an item being defect is $p$. An unbiased estimator for $p$ is $\hat p = \frac{X}{n}$ where $X$ = amount of items ...
6
votes
0answers
88 views

Online estimation of drifting discrete probability

I recently come across (in a practical setting) to the following problem. Suppose I receive items from a finite set ,one at a time . At each moment one item is drawn independently from an unknown ...
2
votes
0answers
64 views

Laplace Method - Estimation of an integral

I am just working on a paper by Shinzo Watanabe on "Asymptotic Evaluations of Wiener Functional expactations": $μ(dx)$ is the $d$-dimensional Gaussian distributions So that is the interesting ...
0
votes
0answers
29 views

Sequential Hypothesis Test

Consider a random variable Y and the pair of simple hypotheses: $H_0 : Y \sim N(0,10)$ $H_1 : Y \sim N(m,10)$ Here $m$ is an unknown mean which is sampled from a distribution $N(0,1)$ and then ...
2
votes
2answers
38 views

generalized concept of “expected value” for outcome sets without a vector space structure

Does the following concept has a name ? Starting point : "expected value" What we call "expected value" for a random variable $X$ requires a vector space structure on its outcome set. Say for some ...
0
votes
1answer
40 views

Calculating a bias given the MOM and MLE

Question: A company manufactured objects, starting from 1 to N. One of the objects is selected, and serial number is 888. We are asked to find things like the MOM, MLE etc. which I managed without ...
1
vote
0answers
31 views

Estimate parameters of Wishart matrix.

Given a sequence of real Wishart matrices $W_1 , \cdots , W_k \sim \mathcal{W}_m(n,\Sigma)$ where $\Sigma$ is a singular matrix. Are there good estimates for the degrees of freedom? The MLE for $\...
1
vote
0answers
26 views

Properties of conditional expectation-tower property

I was reading Prof. Amir Dembo's lecture notes on Stochastic Process. There are two exercises that I cannot figure out. The first exercise is from 2.3.7 of the notes: Let $Var(Y|\mathcal{G})= \...
0
votes
0answers
103 views

Exponential family distribution and sufficient statistic.

The exponential distribution family is defined by pdf of the form: $$ f_x=(x;\theta) = c(\theta) g(x) exp \Big[\sum_{j=1}^l G_j(\theta) T_j(x)]$$ Where $\theta \in \Theta$ and $c(\theta)>0$ And $...
0
votes
1answer
27 views

Exmaple on convergence of random variables - change of integral boundaries in normal distribution

Suppose X$_n$ folowes $N(\mu,\frac{1}{n^2})$. Thus $$ F_{X_n} = \frac {n}{\sqrt{2\pi}}\int_{-\infty}^xexp(-\frac{1}{2}n^2y^2)dy= \frac {n}{\sqrt{2\pi}}\int_{-\infty}^{nx} exp(-\frac{1}{2}z^2)dz$$. ...
0
votes
0answers
31 views

Variance of Linear MMSE estimator from three measurements.

I am here presenting a question on calculating the variance of an Estimator. This is a problem in localization of vehicle using ranging from Vehicular Ad hoc Network and Change in location from ...
1
vote
0answers
28 views

Writing the Extended Kalman Filter

Suppose that you have a system: \begin{cases}\dot{x}(t)=A(u(t))x(t)+d_1(t)\\y(t)=f(x(t))+d_2(t),\end{cases} where $A:\mathbb{R}\longrightarrow\mathbb{R}^{n\times n}$ is a matrix function, $u:\mathbb{R}...
0
votes
1answer
30 views

Estimate how many tracks the city has.(Almost done, but can not find whether estimator is biased or not)

I try to estimate the number of tracks in the city by observing their serial numbers. Assume that the serial numbers are drawn from a uniform probability density ranging from 0 to an unknown parameter ...
0
votes
1answer
369 views

Beta distribution as a member of the exponential family

I come across the beta distribution quite frequently when solving exercises for my statistics class. However, I have not been able to fully grasp how to work with it. Exponential family form is: $$...
3
votes
1answer
87 views

Gamma distribution family and sufficient statistic

Let X $=X_1,...X_n$ be a random sample iid from the probability density function: $$ f(x;\theta)=\frac{\Gamma(\theta)\sin(\pi\theta)\theta^{1-\theta}}{\pi}e^{-\theta x}x^{-\theta}$$ $x>0, 0<\...
1
vote
2answers
29 views

ML estimation with given samples

Let $X_i,...,X_n$ be a random independent sample from a distribution with pdf $$ f(x;\theta)= (\theta + 1)x^{-(\theta+2)},$$ where $x>0$, and $\theta > 0$. What is the ML estimate for the ...
0
votes
1answer
64 views

derive sufficient statistic from a random independent sample from a weibull distribution

Suppose $X_i$ is a random independent sample from a Weibull distribution $$ f(x) = \frac{\beta}{\theta^\beta}x^{\beta-1}\exp\big(-(\frac{x}{\theta})^\beta\big)$$ Find a sufficient statistic for ...
0
votes
1answer
56 views

derive asymptotic distribution of the ML estimator

Let $x$ be a random variable with probability density (pdf) $$f(x)= (\theta +1)x^\theta $$ where $\theta >-1$. The expressions for its mean and variance are $$E(X)= \frac{\theta + 1}{\theta +...
0
votes
1answer
52 views

unbiased pool estimator of variance

I'm not sure I'm calculating the unbiased pooled estimator for the variance correctly. Assuming 2 samples where $\sigma_1 = \sigma_2 = \sigma$ and is uknown, these are my definitions: Sample ...
0
votes
0answers
23 views

need help identifying a formula for “pseudo-entropy”

Maintaining some old code, I've come across: $$\text{pseudo-entropy} = -x \log(x) + x ^{0.45} \cdot (1 - x) ^ {16}$$ I simply need a name for this formula so I can read up on what it's supposed to ...
0
votes
0answers
16 views

Bias in wilson estimator

In case a population is small when estimating proportion p is small and the samle size n is small, with the typical estimator for population proportion $\hat{p_1}=\frac{X}{n}$ one might easily get ...
1
vote
2answers
71 views

Show that $\frac1n(\sum_{i=1}^n\log\frac{1}{1-X_i})^3$ is a sufficient statistic for $\beta$ in a Beta$(\alpha,\beta)$ density

Suppose that $X_1,...,X_n$ From a random sample from a distributio with density $$ f(x) = \frac{\Gamma{(\alpha+\beta)}}{\Gamma{(\alpha)}\Gamma{(\beta)}}x^{\alpha-1}(1-x)^{\beta-1} \quad,\text{ if }\...
0
votes
0answers
39 views

Derive the asymptotic distribution of maximum likelihood estimator

Suppose that $X_1,...,X_n$ is a random sample from a distribution with pdf : $$ f(x,\theta)= \frac{\theta^3}{2}x^2e^{-\theta x} \space for \space 0,x,\infty$$ I found that $l(\theta)=-nlog(2)+3nlog(\...
0
votes
1answer
154 views

Calculating variance of an estimator

Given that $ \operatorname{Var}(x)=\frac{3}{4}\theta^2$, I want t find the variance of estimator $\hat{\theta_1} = \frac{2n}{3}\sum_{i=1}^nX_i$. EDIT: $X_1,...,X_n$ are independent and identically ...
1
vote
1answer
46 views

Find a sufficient statistic. [closed]

Suppose that $X_1,\ldots,X_n$ is a random sample from a distribution with pdf $$ f(x;\theta)=\frac{\theta^3}{2}x^2e^{-\theta x}, \quad 0<x<\infty$$ where $0<\theta<\infty$ Find a ...
0
votes
0answers
75 views

calculate variance of unbiased estimator in Rayleigh distribution

Given : $\hat{\theta} = \frac{\sum X_i^2}{2n} $ and $ E(X^2) = 2\theta $. $\hat{\theta}$ is unbiased estimator for $\theta$ based on i.i.d sample from $f(x;\theta)= \frac{x}{\theta}e^{\frac{-x^2}{2\...
0
votes
0answers
15 views

Induced bias in r.v. A $= \pi R^2$ where R is unbiased measurement of fixed unknown.

I have particularly poor and incomplete notes that refer to induced bias in the random variable A of the area of a circle that is dependent on random variable R which is the measured radius (true ...
0
votes
0answers
39 views

Bounds on the a posteriori error covariance of Kalman filter

I am looking to derive a lower bound for the a posteriori error covarinace ($\bar{\Sigma}$) in discrete-time Kalman filter (in steady state). The bounds on the a priori error covariance ($\Sigma$) ...
1
vote
0answers
69 views

Variance of mean estimator with variable sample size

I'm looking at a random variable that takes vectors $\newcommand{\vv}{\mathbf{v}} \vv_1, \dotsc, \vv_n \in \mathbb{R}^d$ and calculates their average, after applying "blankout" noise to them. So we ...
2
votes
1answer
32 views

Estimation $\mu^2$ under certain conditions.

Let $X_1,X_2,....,X_n$ be a random sample of size $n$ from a population with cdf $F()$. Let $E(X)=\mu$ exist. Then estimate $\mu^2$ unbiasedly for the following three cases:- (i) $Var(X)=\sigma^2$ ...
2
votes
1answer
37 views

Estimator for $\theta$ using the method of moments

Exercise : Using the method of moments, find the estimator for $\theta$ for a random sample $X_1, \dots, X_n$ that follows the distribution with pdf $f(x) = \theta x^{-2}, \; 0 < \theta \leq x &...
0
votes
0answers
9 views

Particle Filters: whether current measurement depends on previous measurement

Recently I was learning basic estimation theory. I am confused that whether the measurement $z_k$ is related to previous measurements $z_{1:k-1}$. A quick recap of the estimation problem. Given ...
0
votes
0answers
11 views

Piece-wise estimators for expectation and variance

The transition probability, $C$, for all pairs of articles $(s,t)$, are mapped to an integer, $d \in D = [0,100]$. $s$ is source article, $t$ is the target article, and $s, t$ are drawn from the same ...
0
votes
1answer
62 views

Proofs for consistency of estimating equations / M-estimators without a compact parameter space?

Most proofs for the consistency of parameters obtained from estimating equations depend on a compact parameter space. However, I have almost never worked with parameter spaces that are compact (they ...
3
votes
1answer
582 views

Maximum Likelihood Estimation for Zero-inflated Poisson distribution

I am trying to do exactly what the title says. What I have is the log-likelihood function as follows: Likelihood function, where $I_i = 1$ when $X_i = 0$, and $I_i = 0$ otherwise. Then I took the ...
0
votes
0answers
15 views

Error of decay of the estimator risk where f is a piecewise Lipschitz function, and gets estimated with piecewise constant function.

I need to find the error of decay expressed in Big O notation, of the estimator risk. When $f$ is Lipschitz smooth the error of decay is $O(n^{-2/3})$. Now I tried the same, but then with the fact ...