Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

1
vote
0answers
12 views

bias reduction when the bias depends on the true parameter

Let's say we estimate a parameter, $\theta$, by $\hat{\theta}$. For this estimator we have the following property that $$\hat{\theta}\to_{p}\theta+f(\theta)$$ where $\to_{p}$ denotes convergence in ...
2
votes
1answer
26 views

How to prove that the maxium likelihood estimator of $\theta$ is aysmptotically unbiased and cosistent

In a class we looked at this example: Let $X_1,...,X_n\sim U(0,\theta)$. Then the maximum likelihood function is $\mathcal{L}(\theta) = \begin{cases} \dfrac{1}{\theta^{n}} & \text{if } ...
0
votes
1answer
54 views

Maximum likelihood estimator and confidence interval

Let $\theta$ be an unknown constant. Let $W_1,…,W_n$ be independent exponential random variables each with parameter $1$. Let $X_i=θ+W_i$. First, I need to find $\hat\theta _{ML}(x_1,\ldots ,x_ n)$. ...
1
vote
0answers
17 views

MLE of variance for a spherical Gaussian

I am trying to implement the X-Means clustering algorithm. In it, the authors use the BIC to determine which model fits the data best. It is explained here: ...
1
vote
1answer
44 views

Sum of variances of multinomial distribution.

I've k fair coins, and I would like to know the number of heads obtained in $n$ trials. But that is simple binomial distribution. But if I want to find out how much it varies from binomial ...
1
vote
1answer
29 views

Dice roll, estimator, epsilon

We roll a non-symmetric die. Let $X_n$ be the reulst of $n$-th roll. $$P(X_n = 6)= \frac{1}{6} + \varepsilon, \ P(X_n = 1) = \frac{1}{6} - \varepsilon, \ P(X_n=2) = ... = P(X_n = 5) = \frac{1}{6} $$ ...
0
votes
0answers
23 views

Most efficient estimator

$X_1,...X_n$ is a random sample of size $n$ from a population with mean $\mu$ and variance $\sigma^2$.There are three estimators for $\mu$:  $\hat\mu _1=\frac{x_1+x_2}{2}$ $\hat\mu ...
0
votes
0answers
16 views

Variance estimation of a diffusion process

The framework of this question is a 1 dimensional diffusion process, defined ny the following equation: $dx_t=adt+bdw_t$ Where $w_t$ is a standard berownian motion and and $a$ is a constant drif ...
0
votes
0answers
4 views

A mix between the Horvitz-Thompson and ordinary estimator

I have asked this question on mathoverflow, but got no answer. Here I have corrected some mistakes and wish to hear any ideas that may bring at least numerical result: The data I have two samples: ...
2
votes
0answers
71 views

Near-Application of Cauchy-Schwarz Inequality

I have the following situation: I have two estimators of $\alpha$, both via maximum likelihood of the density: $$ f(x,y\mid \alpha,\beta) = f(y \mid x,\alpha,\beta)f(x \mid \alpha) $$ One uses only ...
0
votes
2answers
53 views

How to find the bias, variance and MSE of $\hat p$?

If $X_1,\dots,X_n$ are iid $\mathrm{Binomial}(3,p)$, then the maximum likelihood estimator of $p$ is $$\hat p = \frac{1}{n}\sum_i X_i$$ Find the bias, variance and MSE of $\hat p$? We are asked to ...
0
votes
1answer
32 views

linear least square estimation with random sum

Let $N$ be a geometric r.v. with mean $1/p$; let $A1,A2,… $be a sequence of i.i.d. random variables, all independent of $N$, with mean $1$ and variance $1$; let $B1,B2,… $be another sequence of i.i.d. ...
0
votes
0answers
15 views

Second partial derivative of a minimum function

I am reading a book on detection and estimation theory, and the author had this to say in the derivation of the white noise process from the Wiener process: We can formally obtain the covariance ...
0
votes
0answers
4 views

Function of efficient estimator

Say I have an efficient estimator $\theta$, call it $\hat{\theta}$. If I wanted to estimate another quantity, call it $\delta = g(\theta)$, is there any result which allows me to say that ...
2
votes
1answer
125 views

Hypothesis test between two normal distributions

Let $T_1,T_2,\ldots ,T_ n$ be i.i.d. observations, each drawn from a common normal distribution with mean zero. With probability $1/2$ this normal distribution has variance $1$, and with probability ...
0
votes
1answer
67 views

Trajectory estimation

The vertical coordinate (“height") of an object in free fall is described by an equation of the form $x(t) = \theta _0 + \theta _1t + \theta _2 t^2,$ We assume that $\theta_0$ is a known constant. We ...
1
vote
1answer
36 views

Unbiased estimator for the sum of numbers

Let $\alpha_1, \dots, \alpha_n \in \mathbb{R}$. We want to approximate the sum as follows $$ S = \sum_{i=1}^{n} \alpha_i \approx \dfrac{n}{c} \sum_{i=1}^{c} \alpha_i, $$ where $\alpha_i$ is picked ...
1
vote
0answers
25 views

Hammersley–Chapman–Robbins bound for Rice distribution

I am trying to evaluate the Hammersley–Chapman–Robbins bound for the variance of an unbiased estimate $\hat{\alpha}$ of $\alpha$ (for a given $\sigma$) for the Rice distribution: $$p(x|\alpha,\sigma) ...
0
votes
0answers
16 views

Expectation of inverse of a symmetric matrix with gaussian elements

Is there any way to calculate: \begin{equation} \mathbb{E} \; ( H^{T}H )^{-1} \end{equation} assuming that the entries of the matrix $H$ are gaussian random variables with unknown means but same ...
0
votes
0answers
18 views

ML estimator for correlated random vectors

Suppose, we have two random vectors in $\mathbb{R}^2 $ denoted by $(X_1,Y_1)$ and $(X_2,Y_2)$ . The individual random variables can only take value in $\lbrace 0, 1 \rbrace$. We get samples of the ...
1
vote
1answer
47 views

UMVUE for pdf $f_{\theta}(x) = \theta e^{-\theta x}, x>0$

Let $X_1,\ldots,X_n$ be a random sample from a pdf $f_{\theta}(x) = \begin{cases} \theta e^{-\theta x}, & x>0 \\ 0, & \text{otherwise} \end{cases}$, where $\theta>0$ is an unknown ...
1
vote
0answers
17 views

Fisher Expected Information for a Gaussian Process model

Suppose I have a two dimensional Gaussian process model (GP), defined by a squared exponential correlation function s.t: $$R(x_{i},x_{j}) = \exp\left(-\frac{|x_{i} - x_{j}|^2}{2}\right).$$ I am ...
0
votes
0answers
23 views

calculate The maximum likelihood estimator of parameter $\mu$ according to $T$

suppose $X_1,X_2,\ldots,X_n$ be a random sample of $N(\mu,1)$. if $T=\sum_{i=1}^n I_{(X_i<0)}$ how can I calculate The maximum likelihood estimator of parameter $\mu$ according to $T$. ($\Phi$ is ...
0
votes
0answers
10 views

Finding a bayes estimator

Let $X_1,...,X_n|\eta~\exp(1,\eta)$ and $\eta$~$N(\mu,1)$, where $\mu\epsilon\Re$. Find the Bayes estimator $\eta$ under the squared error loss. After finding the joint likelihood of $exp(1,\eta)$ ...
-2
votes
1answer
54 views

calculating $\mathbb E\left(\exp\left(\frac{1}{2}\sum_{i=1}^n X_i^2\right)\right)$ [closed]

suppose $X_1,X_2,\ldots,X_n \sim \mathcal N(0,\sigma^2)$. How can I calculate $$\mathbb E\left(\exp\left(\frac{1}{2}\sum_{i=1}^n X_i^2\right)\right)$$
0
votes
0answers
19 views

calculating UMVUE of parameter $(1-\sigma^2)^-\frac{n}{2}$.

suppose $X_1,X_2,\ldots,X_n$ be random sample of $N(0,\sigma^2)$. how can I calculate UMVUE of parameter $(1-\sigma^2)^-\frac{n}{2}$. I know $T=\sum_{i=1}^n X_i^2$ is Sufficient and complete ...
1
vote
1answer
18 views

How to evaluate the goodness of Fit of parameters obtained from EM algorithm

I have a set of observations $\mathcal{Y} = {Y_1, \ldots, Y_T}$. I am running EM algorithm to fit the observations to the following Hidden Markov Model $$A = [a_{ij}]_{N \times N}, a_{ij} = P(X_{k+1} ...
0
votes
1answer
21 views

Hypothesis Test on Gaussian Mixture

I have data blocks being received at a node which can be presumed to be Gaussian sequence, $X(m_1,sigma_1).$ In some of the blocks a separate Gaussian stream $Y(m_2,sigma_2)$ adds to the original ...
1
vote
1answer
30 views

Compound distribution with unknown distribution of its hyperparameter

Suppose $X\sim \mathcal{N}(0,\sigma)$, and $\sigma$ is another random variable in a sense that we only know that it is some constant random variable with finite support, i-e $\sigma \in [\sigma_\max, ...
0
votes
0answers
14 views

What is the test statistic mentioned in the paper by Dai and Singleton?

I have read the paper with title 'Specification Analysis of Affine Term Structure Models' (2000) by Qiang Dai and Kenneth J. Singleton. On page 22 there's a table which include test statistics, these ...
0
votes
0answers
34 views

probability distribution estimation from correlated samples

I am looking to solve the following estimation problem. Consider a blackbox where (given below) given an input X, its N observations are recorded as output. These observations are denoted by $Y_1, ...
1
vote
0answers
14 views

Rao-Blackwell improvement for a nonrandomized estimator

Context: please consider a parametric statistical model $(\mathcal{Y},\{P_\theta:\theta\in\Theta\})$ and suppose that we are estimating $g(\theta)$. Associated with this is the set of decisions ...
0
votes
1answer
87 views

Estimator for second moment for Poisson random variable

Let $X \sim Poiss(\lambda)$. As, $\displaystyle \sum_{i=1}^{N} X_i $ is sufficient statistic for both mean (and variance) of $Y$, so we can define the unbiased estimate for mean as , $ s=\frac{1}{N} ...
0
votes
0answers
31 views

Determining the actual number of observations in a dataset

I have two datasets one is a dataset with doctors in which I have the procedures they have performed at a given hospital where the actual number of procedures is not captured by this data since it is ...
0
votes
0answers
21 views

What is the nonlinear estimator for Gaussian Random variable?

I know that the best estimator is $g(x)=E\{Y|X=x\}$ and the conditional density for jointly Gaussian random variables is known to be Gaussian with mean and variance given by ...
0
votes
1answer
18 views

Finding an maximum likelihood estimator (Bernouilli problem)

Could someone point me in the right direction? Suppose we compare 2 treatments. For each patient we observe $(Y_i,R_i)$ where $Y_i$ denotes if the treatment was succesfull ($Y_i=1$) or not ...
0
votes
0answers
34 views

Cramer-Rao-Bound for squared parameter

I have a random sample $X_{i}, \ldots, X_{n} \sim \mathcal{N}(\theta,1)$. I would like to find the Cramer-Rao lower bound for the variance of unbiased estimators of $\theta^{2}$. Here's what I have: ...
0
votes
1answer
29 views

estimate coefficients of $y = \alpha x + \beta y + \gamma z + \epsilon$

I know how to find $m$ and $b$ for $y= mx +b$, which is : $m= \frac{\bar{x}\bar{y}- \bar{xy}}{(\bar{x})^2 - \bar{x^2}}$ and $b= \bar{y} - m\bar{x}$ How can we estimate $\alpha, \beta, \gamma$ and ...
2
votes
1answer
42 views

Prove that $\int k(w)o(h^2w^2)dw=o(h^2)$ for $\int k(w)dw=1$

Suppose that $k$ is nonnegative real-valued function satisfying $$ \int k(w)dw=1,\quad\int wk(w)dw=0,\quad\int w^2k(w)dw=\kappa_2<\infty.\tag{$\star$} $$ (The limits of the integrals are all ...
2
votes
0answers
394 views

How to estimate variances for Kalman filter from real sensor measurements without underestimating process noise.

As the title says, I want to estimate the variances needed for a Kalman filter from real sensor measurements only. For example we can take a temperature sensor, but the solution shall be as ...
0
votes
0answers
24 views

Description length in model coding

In class, our professor posted the following: We will discretize $\theta$ (some model) into $1/\sqrt{n}$ distinct values. Intuitive argument: with N data points, our estimation error for $\hat ...
0
votes
1answer
42 views

Maximum likelihood estimator for general multinomial

Let $(X_1,\ldots,X_r)\sim\text{multinomial}(n,(p_1,\ldots,p_r))$, where $p_r=1-p_1-\cdots-p_{r-1}$. The random likelihood is $Ap_1^{X_1}\ldots p_r^{X_r}$, for some non-zero $A$. The random ...
0
votes
1answer
15 views

Computing likelihood for data corrupted by zero mean noise

The following statement is from a text on Statistical Estimation. I am trying to figure out how the likelihood function was arrived at. By definition of likelihood, ...
0
votes
1answer
16 views

Maximum Likelihood (ML) estimation when 1 estimator is dependent on the other.

Let $\mathcal{L}(\theta_1,\theta_2)$ be the log-likelihood function. If I manage to find an estimator for $\theta_1$, as $\hat{\theta}_1=g(\theta_2,data)$. Then, if I want to find a ML estimator for ...
0
votes
0answers
14 views

How can I find the nonlinear and linear MS estimates of y in terms of x and the resulting MS errors?

If $y=x^3$, find the nonlinear and linear MS estimates of $y$ in terms of $x$ and the resulting MS errors? This is what I got for the nonlinear MS estimation: Since $e=E\{[y-C(x)]^2\}$, $C(x)=x^3$ and ...
0
votes
0answers
11 views

how can I find the resulting MS error with linear estimation?

The problem says: If $\eta_x=\eta_y=0, \sigma_x=\sigma_y=4$ and $\hat{y}=0.2x$ (linear estimate), find $E\{(y-\hat{y})^2\}$. I am doing this, based on Papulis formulas for homogeneous linear estimate: ...
0
votes
0answers
17 views

How Can I show that a=A in this linear MS estimation problem?

How can I show that if the constants A,B and a are such that $E\{[y-(Ax+B)]^2\}$ and $E {\{[(y-\eta_y)-a(x-\eta_x)]^2 \}}$ are minimum, then $a=A$. I am trying to use this: $e=e_m$ is minimum if ...
3
votes
0answers
48 views

Trying to show convergence (in probability) of integrals using Taylor expansion

I've been working for a long time now on how to prove a proposition given in a paper about the asymptotic normality of POT-quantile estimators. Hope somebody can help me out. Proposition (i) Let ...
2
votes
0answers
50 views

Simulation Velocity of a harmonic oscillator system

I am write a simulation for get true Velocity of a harmonic oscillator system as Where P=[p1 p2;p2 p3] can find using Rung-Kutta Integration method with P(0)=[1 0; 0 1] This is code to find p Now, ...
0
votes
1answer
33 views

A Estimation about Hölder condition

Let $p:[0,\inf) \to \mathbb{R}$ be a contionous function such that $p(0)=0$ Fix $a>1/2 , k$ is a positive integer $>\frac{1}{a-\frac{1}{2}}$. Suppose for all $n \in \mathbb{N}$ and $\lambda ...