Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

1
vote
2answers
96 views

Let $(X_1…X_n)$ have an exp($\lambda$) distribution. Prove that $\frac{1}{\frac{1}{n}\sum{X_i}}$ is not a unbiased estimator of $\lambda$

the main problem is that i have no clue on calculating $E(\frac{1}{x})$ let $U = \frac{1}{\frac{1}{n}\sum{X_i}}$ then, $E(U) = n*E(\frac{1}{\sum{X_i}})$. I think that i'm supposed to calculate: ...
1
vote
1answer
64 views

Order related to Empirical distribution function and Normal distribution

Let $X_1,\dots,X_n$ are i.i.d with distribution function $F$. Let $\hat F_n$ be its empirical distribution function, i.e., $$ \hat F_n(x)=\frac1n\sum_{i=1}^n1_{\{X_\le x\}}(x) $$ where $1_A(x)$ is the ...
1
vote
2answers
98 views

Estimation theory: Finding MSE and Variance of an Estimator

Let $\hat{A}$ be an estimator of $A$ where $a<A<b$ and $\tilde{A}$ be another estimator such that$$\tilde{A} = \cases{a \text{ if } \hat A \leq a\\\hat A \text{ if } a< \hat A <b\\{b ...
1
vote
2answers
345 views

Show that estimates are unbiased

The following is a problem in my book that I don't really understand: We take a random sample: $x_1,x_2,\ldots,x_n$ from a population that is $N(μ,σ)$ where $\mu$ and $\sigma$ are unknown. We build ...
1
vote
1answer
146 views

Sufficient Estimators and Generalized Likelihood Ratios

If you can make the assumption that a sufficient statistic exists for some parameter - let's call it $\theta$. How would you show that the critical region of a likelihood ratio test will depend on ...
1
vote
2answers
454 views

Question about unbiased estimator

$X_1,X_2,\ldots,X_n$ are iid random variables $B(1,\theta)$ where $0< \theta<1$. Let $w = 1$ if $\sum_i X_i = n$ and $0$ otherwise. What is the best unbiased estimator of $\theta^2$. Attempt: ...
1
vote
1answer
49 views

Conditional expectation and Rao Blackwell

Consider a family of densitites $f(x,\theta)=\frac{\exp(-\sqrt{x})}{\theta}$. Let $X_1$ be a single observation from this family. I have shown that $\sqrt{X_1}/2$ is an unbiased estimator. Now ...
1
vote
2answers
90 views

Finding the MLE of a function when L'($\theta$) doesn't depend on $\theta$

Here's the problem: Find the MLE of of $\theta$ when $f(x\mid\theta)=(1+x\theta)/2$ for $-1<x<1$, $=0$ otherwise. $0<\theta<1$ Find the maximum likelihood of $\theta$ and find its ...
1
vote
1answer
114 views

Optimal combination of two estimates

I have a set of random variables, $X_1,\dots,X_N$. They are i.i.d. Gaussian with zero mean and $w$ variance. I observe $Y_1,\dots,Y_N$ where $Y_i=\sum_{j=1}^N a_{ij} X_j+N_i$ where all $a_{ij}$s are ...
1
vote
2answers
94 views

What exactly is simple consistency?

Now yes, I know the definition: A sequence of estimators $\{T_n\}$of $\tau(\theta)$ are consistent if for every $\epsilon > 0$ $$ \lim_{n\to\infty}P[|T_n-\tau(\theta)|\leq\epsilon]=1 \\\text{ ...
1
vote
1answer
72 views

find $\theta_{MLE}$ for a function

For $$ f(x;\theta)=(\theta+1)x^{-\theta-2} $$ find the maxmimum likelihood estimators (MLEs) for $\theta$ based on a random sample of size $n$. My work so far: $$ \begin{align} \prod_{i=1}^n ...
1
vote
1answer
534 views

Rao-Blackwell unbiased estimator geometric distribution

I'm looking at review questions and having trouble with this one! Let $X_1,\ldots,X_n$ be i.i.d. geometric R.V.s with the pmf: $(1-p)^{x-1}p$, for $x=1,2,\ldots$ and $0<p<1$. I need to use ...
1
vote
2answers
83 views

Filter to obtain MMSE of data from Gaussian vector

Data sampled at two time instances giving bivariate Gaussian vector $X=(X_1,X_2)^T$ with $f(x_1,x_2)=\exp(-(x_1^2+1.8x_1x_2+x_2^2)/0.38)/2\pi \sqrt{0.19}$ Data measured in noisy environment with ...
1
vote
1answer
236 views

How to find parameters from an equation

the question could be "stupid" but i don't know if it is feasible or not, please don't kill me :) EDIT WITH NEW FORMULAS! I have an equation like this: (unfortunatly in my first Q&A i cannot ...
1
vote
3answers
1k views

How to estimate failure probability from count until first failure?

What would be the formula to estimate the rate of failure of some test as a percentage chance of failure from the number of runs of the test until the first failure was seen? For example, considering ...
1
vote
2answers
46 views

how to estimate the phase parameter of a complex function

There is a complex serie: $f(t_n)=\alpha_n+\beta_n i$, for $n = 1,...,N$,$t_n,\alpha_n$ and $\beta_n$ are known.When we have know that $f(t)$ has the following form: $$f(t)=Ae^{-iBt}$$ with unknown ...
1
vote
1answer
94 views

statistics inequality

Let $\theta$ be a discrete pararmeter and $\gamma_{n}$ be an estimator. Prove that for any $c>0$ we have that $$\text{E}[(\gamma_n-\theta)^2] \ge\Pr[|\gamma_n-\theta|>c]\cdot c^2$$
1
vote
1answer
51 views

When is a minimum distance decoder also a maximum likelihood decoder?

It is well known that if we have a binary symmetric channel with crossover probability $\epsilon<0.5$ and we send a word $x$ through it, the most likely word is the one with minimum hamming ...
1
vote
1answer
55 views

Finding the MLE of pareto dist., and trouble interpreting $\prod$ notation properly.

I am generally having trouble understanding how to use product notation when calculating Maximum Likelihood Estimators. The example bellow is from a random sample $X_1,...,X_n$. Find the MLE of ...
1
vote
1answer
108 views

Find the Method Moment Estimator of parameter $\theta$

Find the MME of parameter $\theta$ in the distribution with the density $f(x,\theta)=(\theta +1)x^{-(\theta+2)}$, for $x>1$ and $\theta >0$. So far I think I have a basic understanding of the ...
1
vote
1answer
55 views

Is Sample Covariance Tied to a Specific Distribution

In many sources on data analysis, the author(s) talk about calculating covariance of the data, and the formula is given as such $$ \Sigma = cov(X) = E[(X-E[X])(X-E[X])^T]$$ This formulation is given ...
1
vote
1answer
34 views

How to estimate a vocabulary size?

I have a list of the 1 million most common English words ordered by number of times they appear on all books in Google Books. I want for the user to select from a list of 100 words (small sample) ...
1
vote
1answer
27 views

Conjugate Bayesian analysis

Suppose that conditional on $\tau$, the random variable $X$ has normal distribution with mean zero and variance $1/ \tau$. The prior distribution for $\tau$ is Gamma with parameter $\alpha$ and ...
1
vote
2answers
145 views

Does convergence in probability not imply convergence in distribution for Least Squares estimators?

I have a question relating to convergence in probability and distribution for least squares estimators. Frequently, I see in textbooks that $\hat{\beta} \rightarrow^p b$. Where $b$ is the population ...
1
vote
1answer
196 views

Find maximum likelihood estimator, trick?

Let $Y_1, Y_2, \ldots, Y_n$ iid random variables with density $f(y)=\theta\cdot y^{\theta-1}$, $0<y<1$, $\theta >0$. I need to show that the maximum likelihood estimator of $\theta$ is ...
1
vote
1answer
80 views

Using Neyman pearson lemma when ratio comes out to be zero.

Consider a Bernoulli random variable: $$X_i= \begin{cases} 1, & \text{with probability }p \\ 0, & \text{with probability }1-p \end{cases}$$ You observe the outcomes of two Bernoulli trials ...
1
vote
1answer
239 views

The distribution when combining two samples together?

Suppose $X\sim N(0,{\sigma}^2)$ and $Y\sim N(0,{2\sigma}^2)$ . $X_1, ..., X_m$ are the samples from $X$ and $Y_1, ..., Y_n$ are the samples from $Y$. And then combine two samples as a new sample ...
1
vote
1answer
220 views

Using the MSE criterion, which is a better estimator for $\Theta^2$?

Question: Let $T_1$ and $T_2$ be independent unbiased estimators of a parameter $\Theta$. Assume that $\operatorname{Var}(T_2) = \operatorname{Var}(T_1)$. Using the MSE critertion, define which is a ...
1
vote
1answer
78 views

Proving that the sum of Good-Turing estimators is $1$

I want to know how to go about proving that the Good-Turing estimator has a total probability of $1$. I have seen this proof (page 2) but I found unclear the first step: $$\sum_j \theta[j] = \sum_r ...
1
vote
2answers
437 views

Fast variance calculation

Suppose to have a sequence $X$ of $m$ samples and for each $i^{th}$ sample you want to calculate a local mean $\mu_{X}(i)$ and a local variance $\sigma^2_{X}(i)$ estimation over $n \ll m$ samples of ...
1
vote
1answer
293 views

Gauss-Markov estimator properties

Consider a linear model $$ y = Ab+n, $$ where $b \in \mathbb{R^m}$ is a parameter to be estimated, $n \in \mathbb{R^{n}}$ is a noise with mean $\mathbb{E}n = m_{n}$ and with covariation matrix ...
1
vote
1answer
99 views

How to find out the control function of a cosine wave?

I have a system which is sampling at 100Hz. There is only one input for the system. The output is similar to cosine waveforms with varying frequency. I have no clue how to find out the exact formula ...
1
vote
1answer
294 views

Maximum Likelihood optimal threshold

I have a decision (detection) problem trying to decide between symbols ${0,2}$. I have the two probability density functions: $$ f(z|s=0) = \begin{cases} 0.25z + 0.5, & -2\le\ z <0 \\ ...
1
vote
1answer
132 views

Likelihood function estimating two different means

I completely understand how to find the likelihood functions of simple pdfs. However, how would you attempt to find the likelihood function of a pdf with a negative exponential function with two ...
1
vote
1answer
27 views

Shapiro-Wilk test

I am trying to determine if a given sample comes from a Normal distribution. For that purpose I want to perform a Shapiro-Wilk test in the way stated on wikipedia. My concern comes with the vector ...
1
vote
0answers
59 views

Expectation of $\cos(\|X\|)$ where $X \sim \mathcal{N}(\mu,\Sigma)$

Do: $$ \int_{-\infty}^\infty \int_{-\infty}^\infty \cos\left(\sqrt{x^2+y^2}\right) e^{-\frac{1}{2}\left[\frac{(x-\mu_x)^2}{\sigma_x^2} + ...
1
vote
1answer
57 views

How to compute the unique MLE from an Exponential Family of Distributions?

Let $$ f(x;\theta)=\frac{1}{\pi} \frac{e^{\theta x}\cos(\theta \pi/2)}{\cosh(x)}, x\in{\mathbb{R}} $$ be a family of densities and which is clearly exponential family. Then what is the Maximum ...
1
vote
0answers
26 views

ML estimate of sum of guassian variables?

consider the sum $z=x_{1}+...+x_{k}$, where the scalar variables $x_{i}$ are statistically independent and Gaussian, each having the same mean $0$ and variance $\sigma^2_{x}.$ how can I construct the ...
1
vote
2answers
81 views

Maximum likelihood estimator?

I am looking at some questions from Mods 2010 and I can't figure this one out. I think my problem is technical... We have a sample (L1,R1), ...,(Ln,Rn) with Lj and Rj normally distributed independent ...
1
vote
0answers
25 views

Convergence rate of an estimator

Say we are interested in estimating some unknown real scalar parameter $\alpha$ using data. Suppose the estimator $\widehat \alpha_N$ of $\alpha$ using the data is consistent. I want to know what it ...
1
vote
0answers
67 views

Estimation of a Ito's semi-martingale linear functional

Could someone check my solution for the following problem please? Or maybe propose a smarter/shorter solution. Consider a stochastic process $X=(X_t)_{t \in [0,1]}$ defined in a filtred ...
1
vote
0answers
88 views

Maximum Likelihood Estimator of the exponential function parameter based on Order Statistics

Let $X_1, \ldots, X_n$ be a random sample from the exponential distribution $\exp(\lambda)$. Let $M_n=\max\{X_1, \ldots, X_n\}$ with probability density function $$g_{M_n}(x)=n\lambda e^{-\lambda ...
1
vote
0answers
59 views

Time series (stochastic process) estimating parameters using characteristic function

I have a time series of assets ${A_1, A_2, ..., A_n}$, which is described by a sophisticated distribution having the following characteristic function: $\phi(u; t;\theta)$, where $\theta$ is a vector ...
1
vote
1answer
41 views

Statistics: why is this probablility smaller?

a shipment of goods contains two containers, one container has 300 units and the other container has 700 units. A supervisor checks 30 units in the first container and he finds $X_1$ broken units and ...
1
vote
1answer
41 views

Overestimate of $|\oint_{|z|=R} f(z) \mathrm{d}z|$ with $f(z)=\frac{z^a}{z^2+1}$, $0<|a|<1\mathrm{with} \;a \in \mathbb R$

How can I overestimate, $|\oint_{|z|=R} f(z) \mathrm{d}z |$ with $f(z)=\frac{z^a}{z^2+1}$, $0<|a|<1 \; \mathrm{with} \;a \in \mathbb R$ ? I tried this: $|\oint_{|z|=R} f(z) \mathrm{d}z | <= ...
1
vote
0answers
20 views

A question on Stochastic Approximation

I have just started learning stochastic approximation methods, so the question I'm going to ask may be a trivial one in this field, but I need to know this seriousely. I know, that if $g(x,\xi)$ is a ...
1
vote
1answer
59 views

Is it possible to “customize” the multinomial distribution to your specifications?

So according to the multinomial distribution, the probability function $\Pr(X_1 = x_1, X_2 = x_2, \dots, X_k = x_k)$ is equal to $\dfrac{n!}{x_1! x_2! \cdots x_k!} \cdot p_1^{x_1}\cdot p_2^{x_2} ...
1
vote
1answer
145 views

How to calculate probability using multinomial distribution?

So according to the multinomial distribution, the probability function $\Pr(X_1 = x_1, X_2 = x_2, \dots, X_k = x_k)$ is equal to $\dfrac{n!}{x_1! x_2! \cdots x_k!} \cdot p_1^{x_1}\cdot p_2^{x_2} ...
1
vote
0answers
23 views

ML Estimation for number of animals in a park. Hypothesis Testing.

A park of area $S=10 000 km^2$ was surveyed for bears, and out of $n$ disjoint regions of equal area $s=1km^2$, there were $n_k$ regions with $k=0,1,....,N$ bears. On each of these regions, the amount ...
1
vote
1answer
33 views

Improving related estimates

There are three underlying quantities $x$, $y$, and $a$, where $x$ and $y$ are vectors, and $a$ is a scalar. They are related by $x = ay$. We get noisy observations, $x_0,y_0$. We want to find $a$, ...