Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

2
votes
0answers
74 views

Identification of real functions

this my second question, so I'm still new... thanks in advance for any help! Basically, I'm looking for some references and tools to study the following problem. Consider the following function ...
2
votes
0answers
51 views

Unbiased estimator with conditional expectation.

Suppose that $X$ has a binomial distribution with parameter $N=1$ and $p=1/2$. Y, which is independent of $X$, has a normal distribution with mean $\mu$ and variance 1. Consider the estimator $\mu$ of ...
2
votes
1answer
24 views

How to show this estimation?

i have this polynom $$p(x) = \sum_{i=0}^{m}a_ix^i$$ I want to show, that if $\tilde{z}$ is the approximation to the simple zero digit $z \neq 0$ in first approximation, the following estimation ...
2
votes
2answers
65 views

Why we always put log() before the joint pdf when we use MLE(Maximum likelihood Estimation)?

Maybe this question is simple, but I really need some help. When we use the Maximum Likelihood Estimation(MLE) to estimate the parameters, why we always put the log() before the joint density? To use ...
2
votes
0answers
44 views

What are the properties of median-unbiased estimators?

On Wikipedia it says that " A median-unbiased estimator minimizes the risk with respect to the absolute-deviation loss function, as observed by Laplace." How to prove this? Note that I asked on Cross ...
2
votes
1answer
345 views

Maximum likelihood estimators, hypergeometric and binomial

I'm trying to solve a two part problem. The set up is as follows: consider a bag with $\theta$ red marbles and $7-\theta$ blue marbles, with $\theta$ being unknown. Let $x$ denote the number of red ...
2
votes
0answers
190 views

Fisher Information and minimum variance estimators

I am trying to understand what can be proved about minimum variance estimators. I have changed the question to make it more specific. Let us assume we have some finite set $S$ of elements and we just ...
2
votes
1answer
54 views

Parameter optimization in probabilistic models

Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given ...
2
votes
0answers
93 views

Likelihood Function of Random Process

Given the following data: $$ x(t) = A + \omega(t) $$ where $ \omega(t) $ is an AWGN with zero mean, what would be likelihood function $p(x(t);A)$? I know it could be proven to be: $$ p(x;A) = C ...
1
vote
2answers
103 views

Let $(X_1…X_n)$ have an exp($\lambda$) distribution. Prove that $\frac{1}{\frac{1}{n}\sum{X_i}}$ is not a unbiased estimator of $\lambda$

the main problem is that i have no clue on calculating $E(\frac{1}{x})$ let $U = \frac{1}{\frac{1}{n}\sum{X_i}}$ then, $E(U) = n*E(\frac{1}{\sum{X_i}})$. I think that i'm supposed to calculate: ...
1
vote
2answers
20 views

Variance of sample mean (problems with proof)

Assuming that I have $\{x_1,\ldots, x_N\}$ - an iid (independent identically distributed) sample size $N$ of observations of random variable $\xi$ with unknown mean $m_1$, variance (second central ...
1
vote
2answers
112 views

Estimation theory: Finding MSE and Variance of an Estimator

Let $\hat{A}$ be an estimator of $A$ where $a<A<b$ and $\tilde{A}$ be another estimator such that$$\tilde{A} = \cases{a \text{ if } \hat A \leq a\\\hat A \text{ if } a< \hat A <b\\{b ...
1
vote
1answer
36 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
1
vote
1answer
78 views

Order related to Empirical distribution function and Normal distribution

Let $X_1,\dots,X_n$ are i.i.d with distribution function $F$. Let $\hat F_n$ be its empirical distribution function, i.e., $$ \hat F_n(x)=\frac1n\sum_{i=1}^n1_{\{X_\le x\}}(x) $$ where $1_A(x)$ is the ...
1
vote
2answers
471 views

Show that estimates are unbiased

The following is a problem in my book that I don't really understand: We take a random sample: $x_1,x_2,\ldots,x_n$ from a population that is $N(μ,σ)$ where $\mu$ and $\sigma$ are unknown. We build ...
1
vote
1answer
154 views

Sufficient Estimators and Generalized Likelihood Ratios

If you can make the assumption that a sufficient statistic exists for some parameter - let's call it $\theta$. How would you show that the critical region of a likelihood ratio test will depend on ...
1
vote
2answers
465 views

Question about unbiased estimator

$X_1,X_2,\ldots,X_n$ are iid random variables $B(1,\theta)$ where $0< \theta<1$. Let $w = 1$ if $\sum_i X_i = n$ and $0$ otherwise. What is the best unbiased estimator of $\theta^2$. Attempt: ...
1
vote
1answer
70 views

Conditional expectation and Rao Blackwell

Consider a family of densitites $f(x,\theta)=\frac{\exp(-\sqrt{x})}{\theta}$. Let $X_1$ be a single observation from this family. I have shown that $\sqrt{X_1}/2$ is an unbiased estimator. Now ...
1
vote
2answers
92 views

Finding the MLE of a function when L'($\theta$) doesn't depend on $\theta$

Here's the problem: Find the MLE of of $\theta$ when $f(x\mid\theta)=(1+x\theta)/2$ for $-1<x<1$, $=0$ otherwise. $0<\theta<1$ Find the maximum likelihood of $\theta$ and find its ...
1
vote
1answer
126 views

Optimal combination of two estimates

I have a set of random variables, $X_1,\dots,X_N$. They are i.i.d. Gaussian with zero mean and $w$ variance. I observe $Y_1,\dots,Y_N$ where $Y_i=\sum_{j=1}^N a_{ij} X_j+N_i$ where all $a_{ij}$s are ...
1
vote
2answers
108 views

What exactly is simple consistency?

Now yes, I know the definition: A sequence of estimators $\{T_n\}$of $\tau(\theta)$ are consistent if for every $\epsilon > 0$ $$ \lim_{n\to\infty}P[|T_n-\tau(\theta)|\leq\epsilon]=1 \\\text{ ...
1
vote
1answer
84 views

find $\theta_{MLE}$ for a function

For $$ f(x;\theta)=(\theta+1)x^{-\theta-2} $$ find the maxmimum likelihood estimators (MLEs) for $\theta$ based on a random sample of size $n$. My work so far: $$ \begin{align} \prod_{i=1}^n ...
1
vote
1answer
715 views

Rao-Blackwell unbiased estimator geometric distribution

I'm looking at review questions and having trouble with this one! Let $X_1,\ldots,X_n$ be i.i.d. geometric R.V.s with the pmf: $(1-p)^{x-1}p$, for $x=1,2,\ldots$ and $0<p<1$. I need to use ...
1
vote
2answers
86 views

Filter to obtain MMSE of data from Gaussian vector

Data sampled at two time instances giving bivariate Gaussian vector $X=(X_1,X_2)^T$ with $f(x_1,x_2)=\exp(-(x_1^2+1.8x_1x_2+x_2^2)/0.38)/2\pi \sqrt{0.19}$ Data measured in noisy environment with ...
1
vote
1answer
254 views

How to find parameters from an equation

the question could be "stupid" but i don't know if it is feasible or not, please don't kill me :) EDIT WITH NEW FORMULAS! I have an equation like this: (unfortunatly in my first Q&A i cannot ...
1
vote
3answers
1k views

How to estimate failure probability from count until first failure?

What would be the formula to estimate the rate of failure of some test as a percentage chance of failure from the number of runs of the test until the first failure was seen? For example, considering ...
1
vote
1answer
23 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
1
vote
1answer
99 views

Maximum likelihood estimate vs likelihood ratio tests?

Can someone explain to me the intuition behind why we need likelihood ratio tests. From my understanding, they make use of maximum likelihood estimators over different parameters space and they are a ...
1
vote
2answers
52 views

how to estimate the phase parameter of a complex function

There is a complex serie: $f(t_n)=\alpha_n+\beta_n i$, for $n = 1,...,N$,$t_n,\alpha_n$ and $\beta_n$ are known.When we have know that $f(t)$ has the following form: $$f(t)=Ae^{-iBt}$$ with unknown ...
1
vote
1answer
94 views

statistics inequality

Let $\theta$ be a discrete pararmeter and $\gamma_{n}$ be an estimator. Prove that for any $c>0$ we have that $$\text{E}[(\gamma_n-\theta)^2] \ge\Pr[|\gamma_n-\theta|>c]\cdot c^2$$
1
vote
1answer
63 views

When is a minimum distance decoder also a maximum likelihood decoder?

It is well known that if we have a binary symmetric channel with crossover probability $\epsilon<0.5$ and we send a word $x$ through it, the most likely word is the one with minimum hamming ...
1
vote
1answer
68 views

Finding the MLE of pareto dist., and trouble interpreting $\prod$ notation properly.

I am generally having trouble understanding how to use product notation when calculating Maximum Likelihood Estimators. The example bellow is from a random sample $X_1,...,X_n$. Find the MLE of ...
1
vote
1answer
133 views

Find the Method Moment Estimator of parameter $\theta$

Find the MME of parameter $\theta$ in the distribution with the density $f(x,\theta)=(\theta +1)x^{-(\theta+2)}$, for $x>1$ and $\theta >0$. So far I think I have a basic understanding of the ...
1
vote
1answer
65 views

Is Sample Covariance Tied to a Specific Distribution

In many sources on data analysis, the author(s) talk about calculating covariance of the data, and the formula is given as such $$ \Sigma = cov(X) = E[(X-E[X])(X-E[X])^T]$$ This formulation is given ...
1
vote
1answer
38 views

How to estimate a vocabulary size?

I have a list of the 1 million most common English words ordered by number of times they appear on all books in Google Books. I want for the user to select from a list of 100 words (small sample) ...
1
vote
1answer
27 views

Conjugate Bayesian analysis

Suppose that conditional on $\tau$, the random variable $X$ has normal distribution with mean zero and variance $1/ \tau$. The prior distribution for $\tau$ is Gamma with parameter $\alpha$ and ...
1
vote
2answers
161 views

Does convergence in probability not imply convergence in distribution for Least Squares estimators?

I have a question relating to convergence in probability and distribution for least squares estimators. Frequently, I see in textbooks that $\hat{\beta} \rightarrow^p b$. Where $b$ is the population ...
1
vote
1answer
217 views

Find maximum likelihood estimator, trick?

Let $Y_1, Y_2, \ldots, Y_n$ iid random variables with density $f(y)=\theta\cdot y^{\theta-1}$, $0<y<1$, $\theta >0$. I need to show that the maximum likelihood estimator of $\theta$ is ...
1
vote
1answer
82 views

Using Neyman pearson lemma when ratio comes out to be zero.

Consider a Bernoulli random variable: $$X_i= \begin{cases} 1, & \text{with probability }p \\ 0, & \text{with probability }1-p \end{cases}$$ You observe the outcomes of two Bernoulli trials ...
1
vote
1answer
296 views

The distribution when combining two samples together?

Suppose $X\sim N(0,{\sigma}^2)$ and $Y\sim N(0,{2\sigma}^2)$ . $X_1, ..., X_m$ are the samples from $X$ and $Y_1, ..., Y_n$ are the samples from $Y$. And then combine two samples as a new sample ...
1
vote
1answer
247 views

Using the MSE criterion, which is a better estimator for $\Theta^2$?

Question: Let $T_1$ and $T_2$ be independent unbiased estimators of a parameter $\Theta$. Assume that $\operatorname{Var}(T_2) = \operatorname{Var}(T_1)$. Using the MSE critertion, define which is a ...
1
vote
1answer
81 views

Proving that the sum of Good-Turing estimators is $1$

I want to know how to go about proving that the Good-Turing estimator has a total probability of $1$. I have seen this proof (page 2) but I found unclear the first step: $$\sum_j \theta[j] = \sum_r ...
1
vote
2answers
574 views

Fast variance calculation

Suppose to have a sequence $X$ of $m$ samples and for each $i^{th}$ sample you want to calculate a local mean $\mu_{X}(i)$ and a local variance $\sigma^2_{X}(i)$ estimation over $n \ll m$ samples of ...
1
vote
1answer
334 views

Gauss-Markov estimator properties

Consider a linear model $$ y = Ab+n, $$ where $b \in \mathbb{R^m}$ is a parameter to be estimated, $n \in \mathbb{R^{n}}$ is a noise with mean $\mathbb{E}n = m_{n}$ and with covariation matrix ...
1
vote
1answer
99 views

How to find out the control function of a cosine wave?

I have a system which is sampling at 100Hz. There is only one input for the system. The output is similar to cosine waveforms with varying frequency. I have no clue how to find out the exact formula ...
1
vote
1answer
324 views

Maximum Likelihood optimal threshold

I have a decision (detection) problem trying to decide between symbols ${0,2}$. I have the two probability density functions: $$ f(z|s=0) = \begin{cases} 0.25z + 0.5, & -2\le\ z <0 \\ ...
1
vote
1answer
132 views

Likelihood function estimating two different means

I completely understand how to find the likelihood functions of simple pdfs. However, how would you attempt to find the likelihood function of a pdf with a negative exponential function with two ...
1
vote
1answer
28 views

On the finite expectation of a function

Let $y_i$, $i=1,\dots,n$, be independent Gaussian rv's of mean $\theta_i$ and variance $\sigma^2$ and let $\mathbf{y}:=[y_1,\dots,y_n]^\top$. Consider the function $f\colon \mathbb{R}^n\to ...
1
vote
1answer
40 views

When to use Central Limit Theorem or Cramers Theorem

In for example this paper the authors say The central limit theorem provides an estimate of the probability \begin{align} P\left( \frac{\sum_{i=1}^n X_i - n\mu}{\sigma \sqrt{n}} > x \right) ...
1
vote
0answers
21 views

asymptotic unbiasedness of weibull mle

It's known that the MLEs of the two-parameter Weibull distribution scale and shape parameters are not available in a closed form. It is, however, known that they do exist, are unique, and moreover, ...