Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

1
vote
1answer
250 views

Find maximum likelihood estimator, trick?

Let $Y_1, Y_2, \ldots, Y_n$ iid random variables with density $f(y)=\theta\cdot y^{\theta-1}$, $0<y<1$, $\theta >0$. I need to show that the maximum likelihood estimator of $\theta$ is ...
3
votes
2answers
86 views

Estimating time in harmonic signal

I hope someone can help me with the following problem: Assume a periodic signal of the form $$\begin{align} s(t) &= \sum\limits_{p=1}^P \sin(p\Omega_0t)\\ &= \sum\limits_{p=1}^P ...
0
votes
0answers
81 views

Markov Chain : Montonicity of Sample Mean

Let $\{X_n\}_{n\geq1}$ be an irreducible, ergodic Markov chain with discrete state-space $S$, transition probability matrix $P$ and steady state distribution $\pi = \{\pi_j\}_{j\in S}$. Let $f$ be a ...
1
vote
0answers
25 views

ML Estimation for number of animals in a park. Hypothesis Testing.

A park of area $S=10 000 km^2$ was surveyed for bears, and out of $n$ disjoint regions of equal area $s=1km^2$, there were $n_k$ regions with $k=0,1,....,N$ bears. On each of these regions, the amount ...
0
votes
1answer
84 views

Bounds on least squares and weighted least squares estimator

I was wondering if I can get some help in getting bounds on the parameters estimated by least squares (LS) and weighted least squares (WLS) methods. Suppose our observation model is: $\mathbf{y} = ...
1
vote
1answer
88 views

Using Neyman pearson lemma when ratio comes out to be zero.

Consider a Bernoulli random variable: $$X_i= \begin{cases} 1, & \text{with probability }p \\ 0, & \text{with probability }1-p \end{cases}$$ You observe the outcomes of two Bernoulli trials ...
2
votes
1answer
47 views

Monotonicity of Sample Mean

$X_1,X_2,\ldots$ are drawn i.i.d. from a distribution with mean $\mu$. Define $\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i$ Prove that $\forall t \quad E[|\bar{X}_t - \mu|] \geq E[|\bar{X}_{t+1} - \mu|]$ ...
0
votes
1answer
55 views

Estimator in a one dimensional normal setting with only one observation

Let $X$ have the distribution $N(\theta,1)$ where $\theta \ge 0$. Is $T=X$ an admissible estimator with respect to the mean squared error? Construct an estimator that respects the assumption $\theta ...
6
votes
1answer
6k views

Maximum Likelihood Estimator for Multinomial.

Suppose that 50 measuring scales made by a machine are selected at random from the production of the machine and their lengths and widths are measured. It was found that 45 had both measurements ...
0
votes
1answer
275 views

Finding UMVUE from Lehmann-Sheffe Directly

I am having some trouble with an example from the book I am following. Let $X_1,X_2,...,X_n$ for $n>2$ be an iid set of $N(\mu,\sigma^2)$ random variables with, $\mu\in\mathbb{R}$ and ...
3
votes
2answers
138 views

Estimation of the number of prime numbers in a $b^x$ to $b^{x + 1}$ interval

This is a question I have put to myself a long time ago, although only now am I posting it. The thing is, though there is an infinity of prime numbers, they become more and more scarce the further you ...
1
vote
2answers
155 views

What exactly is simple consistency?

Now yes, I know the definition: A sequence of estimators $\{T_n\}$of $\tau(\theta)$ are consistent if for every $\epsilon > 0$ $$ \lim_{n\to\infty}P[|T_n-\tau(\theta)|\leq\epsilon]=1 \\\text{ ...
4
votes
1answer
155 views

Estimate the scale of $e^{-(m+1) t} \sum _{k=0}^{\infty } \frac{t^k}{k!}\left(\sum _{r=0}^k \frac{t^r}{r!}\right)^{m}$

I would like to estimate the scale of the following series, $$S(m,t)=e^{-(m+1) t} \sum _{k=0}^{\infty } \frac{t^k}{k!}\left(\sum _{r=0}^k \frac{t^r}{r!}\right)^{m},$$ where $e$ is the base of ...
3
votes
1answer
680 views

CRLB/UMVUE estimation of $\theta$

We have a random sample $X_1,X_2,\ldots,X_n$ from a probabilitiy distribution with density $f(x;\theta) = \theta x^{-\theta-1} $ given that $x > 1$, and $0$ else. where $\theta >1 $ is an ...
0
votes
1answer
15 views

Estimating $\hat{p}$

let $X\sim Bin(n,p)$ and $\hat{p} =\frac{X}{n}$ a) Find a constant c such that $E[c\hat{p}(1-\hat{p})]=p(1-p)$ My work: $$ \begin{align} cE[\hat{p}(1-\hat{p})] ...
2
votes
1answer
91 views

Finding the MLE of $\theta$ where $\theta \leq x$

consider the following PDF $$ \begin{eqnarray} f(x;\theta) &=& \left\{\begin{array}{ll} 2\frac{\theta^2}{x^3} & \theta \leqslant x\\ 0 & x< \theta; 0 < \theta ...
1
vote
1answer
90 views

find $\theta_{MLE}$ for a function

For $$ f(x;\theta)=(\theta+1)x^{-\theta-2} $$ find the maxmimum likelihood estimators (MLEs) for $\theta$ based on a random sample of size $n$. My work so far: $$ \begin{align} \prod_{i=1}^n ...
0
votes
1answer
48 views

Random Poisson Sample, Probability in terms of $\vartheta$

If $X_1, X_2, \ldots, X_n$ are a random sample from a Poisson Distribution with mean $\vartheta>0$, how do you find $P(X\le 1)$ in terms of $\vartheta$? I've proven that summing $X_i$ for ...
1
vote
1answer
35 views

Improving related estimates

There are three underlying quantities $x$, $y$, and $a$, where $x$ and $y$ are vectors, and $a$ is a scalar. They are related by $x = ay$. We get noisy observations, $x_0,y_0$. We want to find $a$, ...
3
votes
1answer
467 views

Maximum likelihood estimators, hypergeometric and binomial

I'm trying to solve a two part problem. The set up is as follows: consider a bag with $\theta$ red marbles and $7-\theta$ blue marbles, with $\theta$ being unknown. Let $x$ denote the number of red ...
1
vote
1answer
1k views

Rao-Blackwell unbiased estimator geometric distribution

I'm looking at review questions and having trouble with this one! Let $X_1,\ldots,X_n$ be i.i.d. geometric R.V.s with the pmf: $(1-p)^{x-1}p$, for $x=1,2,\ldots$ and $0<p<1$. I need to use ...
1
vote
0answers
29 views

Is there a way to estimate the range of fitting coefficients from only the data?

Considering an approximation $f$ for a set of $N$ data points $(x,y)$ using, for example, $M$ radial basis functions at arbitrary sites in the domain $f_i = \sum_{j=1} ^M c_j\phi(||x_i-x_j||)$ where ...
1
vote
0answers
90 views

Hypothesis testing problem of Normal distributions.

Consider the following Hypothesis Testing problem: Hypothesis $H_0$ : $X \sim N(\mu_0, \sigma_0)$. Mean $\mu_0$ is known but only upper and lower bounds on $\sigma_0$ are known. Hypothesis $H_1$ : ...
0
votes
1answer
129 views

Finding expected value of variance estimator (sum expansion problem)

I am trying to show that variance estimator $\frac{1}{n}\sum_{i=1}^{n}(X_i-\bar{X})^2$ is biased. I have an example in the book, and there is one step of this derivation I cannot understand: ...
0
votes
1answer
4k views

Calculating the variance of an estimator (unclear on one step)

How can you go from $4V(\bar X)$ to $\displaystyle \frac{4}{n}V(X_1)$? I understand the rest of the steps...
0
votes
1answer
270 views

Can we compute confidence intervals for the variance of an unknown distributions from sample variances?

Assume $X_1,\ldots,X_n$ are i.i.d. with unknown distribution $\mathcal D$ - we only know it is not normal and has finite variance. Is there a way to give confidence intervals for the variance of ...
1
vote
1answer
197 views

Use Held-out estimator for unseen events

Held out estimator is an empiric estimation technique for calculating probabilities of events. (I would put a Wikipedia link, but i couldn't find a wikipedia page on this subject) The main idea is to ...
1
vote
1answer
279 views

Numerical calculation of fisher information

I am trying to obtain numerically the fisher information. Given a likelihood function $$ f(X,\theta),$$ with $X \in [0,1]$. The fisher information is given by $$ ...
0
votes
1answer
263 views

Empirical Bayes estimator for a Beta-Binomial parameters

Let $X_t$ be collected from a Binomial distribution with parameters $N_t$ and $P_t$, where $N_t$ is known for $t= 1, 2, \dots , T$. On the other hand, $P_t \sim \operatorname{Beta}(\alpha_t, ...
0
votes
1answer
262 views

Scale Median for MRE Estimators with Absolute Difference Error Function for Scale Families

Lehmann, in Theory of Point Estimation p.212, defines scale median as the solution to: $${E(X)I(X\le c)} = {E(X)I(X\ge c)}$$ given $X$ is a positive random variable, and ${E(X)}< \infty$. Now ...
0
votes
1answer
117 views

Efficient method of approximating a distribution with Gaussian

Given a univariate uni-modal density function $f(x)$ (very hard to compute its cumulative distribution function (CDF) $F(x)$, not to mention its inverse CDF $F^{-1}(x)$), how to find the best ...
0
votes
1answer
91 views

Clasification of parameter estimation method

Consider that $P$ is the water pressure coming out from a valve A, therefore, the population is all the valve A pressure values. Let $P_{dif}$ be defined as the difference between the maximum and the ...
0
votes
1answer
206 views

Finding the efficiency of an unbiased estimator

I have a random sample drawn from a $N(\theta,\sigma^2)$ distribution with $\sigma^2$ known. I am trying to estimate $\theta$. I need to calculate the efficiency of the unbiased estimator, ...
0
votes
1answer
100 views

How to estimate parameters of a normal distribution?

Suppose one knew that 105 workers were evaluated by their boss. Such evaluation is distributed according to a normal distribution with mean $\mu$ and std. deviation $\sigma$. We also know that 20 ...
-1
votes
1answer
448 views

How to estimate parameters of a uniform distribution?

I have information of the order in which students were classified in regard to their scores in a SAT test. I know the distribution of scores for each student is uniform with support [a,b]. I also know ...
2
votes
1answer
2k views

How do I find the MLE of $\theta$ when x is dependent on $\theta$?

Let $X_{1},X_{2},...,X_{n}$ represent a random sample from a distribution with pdf: $f(x; \theta)=e^{-(x-\theta)}, \theta \le x<\infty, -\infty<\theta<\infty$ | zero elsewhere I need to ...
0
votes
1answer
30 views

How to obtain estimate of covariance matrix that will be guarantee to be semi-positive define?

How to obtain estimate of covariance matrix that will be guarantee to be semi-positive define ? (Is CrossValidated better place for this question ?)
8
votes
2answers
488 views

Why should Gaussian noise have fractal dimension of 1.5?

In a paper I'm trying to understand, the following time series is generated as "simulated data": $$Y(i)=\sum_{j=1}^{1000+i}Z(j) \:\:\: ; \:\:\: (i=1,2,...,N)$$ where $Z(j)$ is a Gaussian noise with ...
1
vote
1answer
389 views

The distribution when combining two samples together?

Suppose $X\sim N(0,{\sigma}^2)$ and $Y\sim N(0,{2\sigma}^2)$ . $X_1, ..., X_m$ are the samples from $X$ and $Y_1, ..., Y_n$ are the samples from $Y$. And then combine two samples as a new sample ...
16
votes
3answers
5k views

Intuitive explanation of a definition of the Fisher information

I'm studying statistics. When I read the textbook about Fisher Information, I couldn't understand why the Fisher Information is defined like this: $$I(\theta)=E_\theta\left[-\frac{\partial^2 ...
2
votes
2answers
537 views

Expected value of a max

We have a roulette with the circumference $a$. We spin the roulette 10 times and we measure 10 distances, $x_1,\ldots,x_{10}$, from a predefined zero-point. We can assume that those distances are ...
1
vote
2answers
792 views

Show that estimates are unbiased

The following is a problem in my book that I don't really understand: We take a random sample: $x_1,x_2,\ldots,x_n$ from a population that is $N(μ,σ)$ where $\mu$ and $\sigma$ are unknown. We build ...
0
votes
1answer
415 views

how can I get minimum error probability for this decision problem?

I have the decision problem for 4 hypotheses as follows: $$H_j: Y_k=N_k-s_{jk},\ k=1,2,\ldots,n;\ j=0,1,2,3.$$ where signals are $s_{jk}=E_0\sin(w_cT(k-1)+(j+\frac{1}{2})\frac{\pi}{2}).$ $$$$ In ...
3
votes
0answers
120 views

Estimate number of distinct items

I have a large array of $n$ integers, some of which may be repeated, and I want to estimate how many distinct integers are in the array. Say the number of distinct integers is $N$. I can sample with ...
2
votes
0answers
206 views

Fisher Information and minimum variance estimators

I am trying to understand what can be proved about minimum variance estimators. I have changed the question to make it more specific. Let us assume we have some finite set $S$ of elements and we just ...
1
vote
1answer
86 views

biasedness/unbiasedness of an MLE.

To show whether an MLE I just found is biased/unbiased, would I need to find the expectation of the answer? Plus would I do this by integrating $\text{MLE} \cdot \text{pdf}$. My MLE is $ ...
0
votes
1answer
69 views

Several Unbiased Estimators

If I have some data set $ D={X_1,...X_N} $ and have an esitmator be "pick the first point" $X_1$, how can I show that this estimator is unbiased? I also have to show why its highly undesirable, and I ...
2
votes
1answer
54 views

Parameter optimization in probabilistic models

Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given ...
1
vote
0answers
79 views

Estimating the number of observations from a set of samples

I repeatedly measure a value $S_n$ which is the sum of a set of $n$ hidden inputs. The goal is to identify the number of hidden inputs. All of the hidden inputs are driven by an experimenter ...
0
votes
0answers
53 views

Identification of parameters problem

I always struggle to get the true essence of identification in econometrics. I know that we state that a parameter (say $\hat{\theta}$) can be identified if by simply looking at its (joint) ...