Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

2
votes
0answers
185 views

Fisher Information and minimum variance estimators

I am trying to understand what can be proved about minimum variance estimators. I have changed the question to make it more specific. Let us assume we have some finite set $S$ of elements and we just ...
2
votes
1answer
54 views

Parameter optimization in probabilistic models

Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given ...
2
votes
0answers
93 views

Likelihood Function of Random Process

Given the following data: $$ x(t) = A + \omega(t) $$ where $ \omega(t) $ is an AWGN with zero mean, what would be likelihood function $p(x(t);A)$? I know it could be proven to be: $$ p(x;A) = C ...
1
vote
2answers
99 views

Let $(X_1…X_n)$ have an exp($\lambda$) distribution. Prove that $\frac{1}{\frac{1}{n}\sum{X_i}}$ is not a unbiased estimator of $\lambda$

the main problem is that i have no clue on calculating $E(\frac{1}{x})$ let $U = \frac{1}{\frac{1}{n}\sum{X_i}}$ then, $E(U) = n*E(\frac{1}{\sum{X_i}})$. I think that i'm supposed to calculate: ...
1
vote
2answers
104 views

Estimation theory: Finding MSE and Variance of an Estimator

Let $\hat{A}$ be an estimator of $A$ where $a<A<b$ and $\tilde{A}$ be another estimator such that$$\tilde{A} = \cases{a \text{ if } \hat A \leq a\\\hat A \text{ if } a< \hat A <b\\{b ...
1
vote
1answer
75 views

Order related to Empirical distribution function and Normal distribution

Let $X_1,\dots,X_n$ are i.i.d with distribution function $F$. Let $\hat F_n$ be its empirical distribution function, i.e., $$ \hat F_n(x)=\frac1n\sum_{i=1}^n1_{\{X_\le x\}}(x) $$ where $1_A(x)$ is the ...
1
vote
2answers
364 views

Show that estimates are unbiased

The following is a problem in my book that I don't really understand: We take a random sample: $x_1,x_2,\ldots,x_n$ from a population that is $N(μ,σ)$ where $\mu$ and $\sigma$ are unknown. We build ...
1
vote
1answer
148 views

Sufficient Estimators and Generalized Likelihood Ratios

If you can make the assumption that a sufficient statistic exists for some parameter - let's call it $\theta$. How would you show that the critical region of a likelihood ratio test will depend on ...
1
vote
2answers
455 views

Question about unbiased estimator

$X_1,X_2,\ldots,X_n$ are iid random variables $B(1,\theta)$ where $0< \theta<1$. Let $w = 1$ if $\sum_i X_i = n$ and $0$ otherwise. What is the best unbiased estimator of $\theta^2$. Attempt: ...
1
vote
1answer
60 views

Conditional expectation and Rao Blackwell

Consider a family of densitites $f(x,\theta)=\frac{\exp(-\sqrt{x})}{\theta}$. Let $X_1$ be a single observation from this family. I have shown that $\sqrt{X_1}/2$ is an unbiased estimator. Now ...
1
vote
2answers
90 views

Finding the MLE of a function when L'($\theta$) doesn't depend on $\theta$

Here's the problem: Find the MLE of of $\theta$ when $f(x\mid\theta)=(1+x\theta)/2$ for $-1<x<1$, $=0$ otherwise. $0<\theta<1$ Find the maximum likelihood of $\theta$ and find its ...
1
vote
1answer
117 views

Optimal combination of two estimates

I have a set of random variables, $X_1,\dots,X_N$. They are i.i.d. Gaussian with zero mean and $w$ variance. I observe $Y_1,\dots,Y_N$ where $Y_i=\sum_{j=1}^N a_{ij} X_j+N_i$ where all $a_{ij}$s are ...
1
vote
2answers
98 views

What exactly is simple consistency?

Now yes, I know the definition: A sequence of estimators $\{T_n\}$of $\tau(\theta)$ are consistent if for every $\epsilon > 0$ $$ \lim_{n\to\infty}P[|T_n-\tau(\theta)|\leq\epsilon]=1 \\\text{ ...
1
vote
1answer
75 views

find $\theta_{MLE}$ for a function

For $$ f(x;\theta)=(\theta+1)x^{-\theta-2} $$ find the maxmimum likelihood estimators (MLEs) for $\theta$ based on a random sample of size $n$. My work so far: $$ \begin{align} \prod_{i=1}^n ...
1
vote
1answer
561 views

Rao-Blackwell unbiased estimator geometric distribution

I'm looking at review questions and having trouble with this one! Let $X_1,\ldots,X_n$ be i.i.d. geometric R.V.s with the pmf: $(1-p)^{x-1}p$, for $x=1,2,\ldots$ and $0<p<1$. I need to use ...
1
vote
2answers
84 views

Filter to obtain MMSE of data from Gaussian vector

Data sampled at two time instances giving bivariate Gaussian vector $X=(X_1,X_2)^T$ with $f(x_1,x_2)=\exp(-(x_1^2+1.8x_1x_2+x_2^2)/0.38)/2\pi \sqrt{0.19}$ Data measured in noisy environment with ...
1
vote
1answer
246 views

How to find parameters from an equation

the question could be "stupid" but i don't know if it is feasible or not, please don't kill me :) EDIT WITH NEW FORMULAS! I have an equation like this: (unfortunatly in my first Q&A i cannot ...
1
vote
3answers
1k views

How to estimate failure probability from count until first failure?

What would be the formula to estimate the rate of failure of some test as a percentage chance of failure from the number of runs of the test until the first failure was seen? For example, considering ...
1
vote
1answer
36 views

Maximum likelihood estimate vs likelihood ratio tests?

Can someone explain to me the intuition behind why we need likelihood ratio tests. From my understanding, they make use of maximum likelihood estimators over different parameters space and they are a ...
1
vote
2answers
49 views

how to estimate the phase parameter of a complex function

There is a complex serie: $f(t_n)=\alpha_n+\beta_n i$, for $n = 1,...,N$,$t_n,\alpha_n$ and $\beta_n$ are known.When we have know that $f(t)$ has the following form: $$f(t)=Ae^{-iBt}$$ with unknown ...
1
vote
1answer
94 views

statistics inequality

Let $\theta$ be a discrete pararmeter and $\gamma_{n}$ be an estimator. Prove that for any $c>0$ we have that $$\text{E}[(\gamma_n-\theta)^2] \ge\Pr[|\gamma_n-\theta|>c]\cdot c^2$$
1
vote
1answer
54 views

When is a minimum distance decoder also a maximum likelihood decoder?

It is well known that if we have a binary symmetric channel with crossover probability $\epsilon<0.5$ and we send a word $x$ through it, the most likely word is the one with minimum hamming ...
1
vote
1answer
58 views

Finding the MLE of pareto dist., and trouble interpreting $\prod$ notation properly.

I am generally having trouble understanding how to use product notation when calculating Maximum Likelihood Estimators. The example bellow is from a random sample $X_1,...,X_n$. Find the MLE of ...
1
vote
1answer
111 views

Find the Method Moment Estimator of parameter $\theta$

Find the MME of parameter $\theta$ in the distribution with the density $f(x,\theta)=(\theta +1)x^{-(\theta+2)}$, for $x>1$ and $\theta >0$. So far I think I have a basic understanding of the ...
1
vote
1answer
60 views

Is Sample Covariance Tied to a Specific Distribution

In many sources on data analysis, the author(s) talk about calculating covariance of the data, and the formula is given as such $$ \Sigma = cov(X) = E[(X-E[X])(X-E[X])^T]$$ This formulation is given ...
1
vote
1answer
37 views

How to estimate a vocabulary size?

I have a list of the 1 million most common English words ordered by number of times they appear on all books in Google Books. I want for the user to select from a list of 100 words (small sample) ...
1
vote
1answer
27 views

Conjugate Bayesian analysis

Suppose that conditional on $\tau$, the random variable $X$ has normal distribution with mean zero and variance $1/ \tau$. The prior distribution for $\tau$ is Gamma with parameter $\alpha$ and ...
1
vote
2answers
153 views

Does convergence in probability not imply convergence in distribution for Least Squares estimators?

I have a question relating to convergence in probability and distribution for least squares estimators. Frequently, I see in textbooks that $\hat{\beta} \rightarrow^p b$. Where $b$ is the population ...
1
vote
1answer
205 views

Find maximum likelihood estimator, trick?

Let $Y_1, Y_2, \ldots, Y_n$ iid random variables with density $f(y)=\theta\cdot y^{\theta-1}$, $0<y<1$, $\theta >0$. I need to show that the maximum likelihood estimator of $\theta$ is ...
1
vote
1answer
80 views

Using Neyman pearson lemma when ratio comes out to be zero.

Consider a Bernoulli random variable: $$X_i= \begin{cases} 1, & \text{with probability }p \\ 0, & \text{with probability }1-p \end{cases}$$ You observe the outcomes of two Bernoulli trials ...
1
vote
1answer
254 views

The distribution when combining two samples together?

Suppose $X\sim N(0,{\sigma}^2)$ and $Y\sim N(0,{2\sigma}^2)$ . $X_1, ..., X_m$ are the samples from $X$ and $Y_1, ..., Y_n$ are the samples from $Y$. And then combine two samples as a new sample ...
1
vote
1answer
223 views

Using the MSE criterion, which is a better estimator for $\Theta^2$?

Question: Let $T_1$ and $T_2$ be independent unbiased estimators of a parameter $\Theta$. Assume that $\operatorname{Var}(T_2) = \operatorname{Var}(T_1)$. Using the MSE critertion, define which is a ...
1
vote
1answer
79 views

Proving that the sum of Good-Turing estimators is $1$

I want to know how to go about proving that the Good-Turing estimator has a total probability of $1$. I have seen this proof (page 2) but I found unclear the first step: $$\sum_j \theta[j] = \sum_r ...
1
vote
2answers
464 views

Fast variance calculation

Suppose to have a sequence $X$ of $m$ samples and for each $i^{th}$ sample you want to calculate a local mean $\mu_{X}(i)$ and a local variance $\sigma^2_{X}(i)$ estimation over $n \ll m$ samples of ...
1
vote
1answer
301 views

Gauss-Markov estimator properties

Consider a linear model $$ y = Ab+n, $$ where $b \in \mathbb{R^m}$ is a parameter to be estimated, $n \in \mathbb{R^{n}}$ is a noise with mean $\mathbb{E}n = m_{n}$ and with covariation matrix ...
1
vote
1answer
99 views

How to find out the control function of a cosine wave?

I have a system which is sampling at 100Hz. There is only one input for the system. The output is similar to cosine waveforms with varying frequency. I have no clue how to find out the exact formula ...
1
vote
1answer
309 views

Maximum Likelihood optimal threshold

I have a decision (detection) problem trying to decide between symbols ${0,2}$. I have the two probability density functions: $$ f(z|s=0) = \begin{cases} 0.25z + 0.5, & -2\le\ z <0 \\ ...
1
vote
1answer
132 views

Likelihood function estimating two different means

I completely understand how to find the likelihood functions of simple pdfs. However, how would you attempt to find the likelihood function of a pdf with a negative exponential function with two ...
1
vote
0answers
26 views

Working with the sum of two independent random variables, and estimating a parameter

A network source sends a sequence of zeros and ones, $X_1, X_2, ...$ with $X_i$(iid) Bernoulli with $p = P(X_i = 1), 0 < p < 1$. Due to disturbances the received sequence is $Y_1, Y_2, ...$ ...
1
vote
1answer
29 views

Shapiro-Wilk test

I am trying to determine if a given sample comes from a Normal distribution. For that purpose I want to perform a Shapiro-Wilk test in the way stated on wikipedia. My concern comes with the vector ...
1
vote
0answers
60 views

Expectation of $\cos(\|X\|)$ where $X \sim \mathcal{N}(\mu,\Sigma)$

Do: $$ \int_{-\infty}^\infty \int_{-\infty}^\infty \cos\left(\sqrt{x^2+y^2}\right) e^{-\frac{1}{2}\left[\frac{(x-\mu_x)^2}{\sigma_x^2} + ...
1
vote
0answers
26 views

ML estimate of sum of guassian variables?

consider the sum $z=x_{1}+...+x_{k}$, where the scalar variables $x_{i}$ are statistically independent and Gaussian, each having the same mean $0$ and variance $\sigma^2_{x}.$ how can I construct the ...
1
vote
2answers
81 views

Maximum likelihood estimator?

I am looking at some questions from Mods 2010 and I can't figure this one out. I think my problem is technical... We have a sample (L1,R1), ...,(Ln,Rn) with Lj and Rj normally distributed independent ...
1
vote
0answers
26 views

Convergence rate of an estimator

Say we are interested in estimating some unknown real scalar parameter $\alpha$ using data. Suppose the estimator $\widehat \alpha_N$ of $\alpha$ using the data is consistent. I want to know what it ...
1
vote
0answers
68 views

Estimation of a Ito's semi-martingale linear functional

Could someone check my solution for the following problem please? Or maybe propose a smarter/shorter solution. Consider a stochastic process $X=(X_t)_{t \in [0,1]}$ defined in a filtred ...
1
vote
0answers
93 views

Maximum Likelihood Estimator of the exponential function parameter based on Order Statistics

Let $X_1, \ldots, X_n$ be a random sample from the exponential distribution $\exp(\lambda)$. Let $M_n=\max\{X_1, \ldots, X_n\}$ with probability density function $$g_{M_n}(x)=n\lambda e^{-\lambda ...
1
vote
0answers
62 views

Time series (stochastic process) estimating parameters using characteristic function

I have a time series of assets ${A_1, A_2, ..., A_n}$, which is described by a sophisticated distribution having the following characteristic function: $\phi(u; t;\theta)$, where $\theta$ is a vector ...
1
vote
1answer
41 views

Statistics: why is this probablility smaller?

a shipment of goods contains two containers, one container has 300 units and the other container has 700 units. A supervisor checks 30 units in the first container and he finds $X_1$ broken units and ...
1
vote
1answer
41 views

Overestimate of $|\oint_{|z|=R} f(z) \mathrm{d}z|$ with $f(z)=\frac{z^a}{z^2+1}$, $0<|a|<1\mathrm{with} \;a \in \mathbb R$

How can I overestimate, $|\oint_{|z|=R} f(z) \mathrm{d}z |$ with $f(z)=\frac{z^a}{z^2+1}$, $0<|a|<1 \; \mathrm{with} \;a \in \mathbb R$ ? I tried this: $|\oint_{|z|=R} f(z) \mathrm{d}z | <= ...
1
vote
0answers
22 views

A question on Stochastic Approximation

I have just started learning stochastic approximation methods, so the question I'm going to ask may be a trivial one in this field, but I need to know this seriousely. I know, that if $g(x,\xi)$ is a ...