Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

11
votes
2answers
683 views

You see a route 14 bus on the moon. What is the most likely number of bus routes on the moon?

This question was asked on a forum and while many argued that the answer is 14 (since the probability of you seeing bus 14 is maximum in this case), I argued against it that they were working ...
11
votes
3answers
3k views

Intuitive explanation of a definition of the Fisher information

I'm studying statistics. When I read the textbook about Fisher Information, I couldn't understand why the Fisher Information is defined like this: $$I(\theta)=E_\theta\left[-\frac{\partial^2 ...
8
votes
2answers
327 views

Why should Gaussian noise have fractal dimension of 1.5?

In a paper I'm trying to understand, the following time series is generated as "simulated data": $$Y(i)=\sum_{j=1}^{1000+i}Z(j) \:\:\: ; \:\:\: (i=1,2,...,N)$$ where $Z(j)$ is a Gaussian noise with ...
7
votes
1answer
2k views

Minimum variance unbiased estimator for scale parameter of a certain gamma distribution

Let $X_1, X_2, ..., X_n$ be a random sample from a distribution with p.d.f., $$f(x;\theta)=\theta^2xe^{-x\theta} ; 0<x<\infty, \theta>0$$ Obtain minimum variance unbiased estimator of ...
5
votes
3answers
267 views

Estimating population size

Let's suppose there are $n$ real numbers $a_0 < ... < a_n$ uniformly selected from interval [0, 1). If one knows $k$ numbers on consecutive positions $a_i < ... < a_{i+k-1}$ how good is ...
5
votes
1answer
188 views

Finding the joint distribution of $X_{1:n}$ and $\overline{X}$

I need to show that, given a random sample of independent variables $X_1, ... , X_n$, each following a distribution EXP($\theta$,$\eta$), that is, ...
4
votes
2answers
5k views

Difference between logarithm of an expectation value and expectation value of a logarithm

Assuming I have a always positive random variable $X$, $X \in \mathbb{R}$, $X > 0$. Then I am now interested in the difference between the following two expectation values: $E \left[ \ln X ...
4
votes
3answers
950 views

Estimating the Gamma function to high precision efficiently?

I know there are several approximations of the Gamma function that provide decent approximations of this function. I was wondering, how can I efficiently estimate specific values of the Gamma ...
4
votes
2answers
319 views

estimate the perimeter of the island

I'm assigned a task involving solving a problem that can be described as follows: Suppose I'm driving a car around a lake. In the lake there is an island of irregular shape. I have a GPS with me in ...
4
votes
1answer
141 views

Showing that $X_{1:n}$ is sufficient for $\eta$, by factorization

I'm asked to show that $X_{1:n}$ (the minimum order statistic) is sufficient for $\eta$, in the case of a random sample $(X_1, ... , X_n)$ where $X_i\sim EXP(1,\eta$) (this is the two-parameter ...
4
votes
2answers
60 views

estimation of a parameter

The question is: $x_i = \alpha + \omega_i, $ for $i = 1, \ldots, n.$ where $\alpha$ is a non-zero constant, but unknown, parameter to be estimated, and $\omega_i$ are uncorrelated, zero_mean, ...
4
votes
1answer
148 views

Estimate the scale of $e^{-(m+1) t} \sum _{k=0}^{\infty } \frac{t^k}{k!}\left(\sum _{r=0}^k \frac{t^r}{r!}\right)^{m}$

I would like to estimate the scale of the following series, $$S(m,t)=e^{-(m+1) t} \sum _{k=0}^{\infty } \frac{t^k}{k!}\left(\sum _{r=0}^k \frac{t^r}{r!}\right)^{m},$$ where $e$ is the base of ...
4
votes
1answer
144 views

MLE of the mean of a heteroscedastic Gaussian time series

Suppose we observe $Y_i\sim \mathcal{N}(\theta_0 + \theta_1 x_i, \sigma_i^2)$, with $x_i$ and $\sigma_i^2$ known for all $i = 1,\ldots,n$ and $Y_1,\ldots,Y_n$ independent. Assume $\theta_0$ is ...
3
votes
2answers
869 views

Inverse problem from pdes

A linear inverse problem is given by: $\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$ where d: observed data, A: theory operator, m: unknown model and e: error. To minimize the effect of the noise; ...
3
votes
2answers
116 views

Estimation of the number of prime numbers in a $b^x$ to $b^{x + 1}$ interval

This is a question I have put to myself a long time ago, although only now am I posting it. The thing is, though there is an infinity of prime numbers, they become more and more scarce the further you ...
3
votes
2answers
358 views

How can I compare two Markov processes?

There is a discrete-time irreductible Markov process with $r$ possible states. $k$ observations were performed. At each observation a state of process was determined. $T_0 = \lbrace 0,1,\dots ...
3
votes
1answer
85 views

Is a probability density function necessarily a $L^2$ function?

If a nonnegative continuous real valued function $f$ is integrable over $\mathbb{R}$ with $$\int_\mathbb{R} f\,\mathrm{d}x = 1,$$ does it hold true $$\int_\mathbb{R} f^2 \,\mathrm{d}x<\infty?$$ ...
3
votes
1answer
411 views

CRLB/UMVUE estimation of $\theta$

We have a random sample $X_1,X_2,\ldots,X_n$ from a probabilitiy distribution with density $f(x;\theta) = \theta x^{-\theta-1} $ given that $x > 1$, and $0$ else. where $\theta >1 $ is an ...
3
votes
3answers
4k views

Prove the sample variance is an unbiased estimator

I'm trying to prove that the sample variance is an unbiased estimator. I know that I need to find the expected value of the sample variance estimator $$\sum_i\frac{(M_i - \bar{M})^2}{n-1}$$ but I get ...
3
votes
1answer
2k views

Maximum Likelihood Estimator for Multinomial.

Suppose that 50 measuring scales made by a machine are selected at random from the production of the machine and their lengths and widths are measured. It was found that 45 had both measurements ...
3
votes
2answers
1k views

biased Maximum Likelihood estimation

Given $N$ points ($x_k$, k from $1$ to $N$) generated from a normal distribution (1-dimensional case) with known mean $\mu$, the Maximum Likelihood estimation of the variance is ...
3
votes
1answer
47 views

Finding an efficient estimator for $\theta$ in $U[0, \theta]$ in terms of the sample maximum

This question appeared in a past exam paper, in the form: Let $X = (X_1\dotsc X_n)\in\mathbb{R}^n$ be an i.i.d. sample from $U[0, \theta], \theta>0$ Apply Rao-Blackwell's theorem to the unbiased ...
3
votes
2answers
69 views

Calculating likelihood of event based on retrospective analysis

I have a simple dataset consisting of the dates/times at which certain medications were taken by a patient. By looking retrospectively I'd like to make a best guess estimate as to which medication ...
3
votes
1answer
495 views

Unstable linear inverse problem: which “dampening” Tikhonov matrix should I use?

A linear inverse problem is given by: $\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$ where d: observed data, A: theory operator, m: unknown model and e: error. The Least Square Error (LSE) model ...
3
votes
1answer
179 views

Solving perturbed polynomial equations

Rather than asking the most general question possible, I will frame it in terms of what I believe is an illustrative example. Let $\epsilon>0$ be a small parameter, let $a,b>0$ and $x\in ...
3
votes
1answer
66 views

How is the “cooking” done in surveys

In my country there's an official center undertaking surveys of voting intention every 4 months. However, they provide only "direct" voting intention, and the statistics obtained are usually pretty ...
3
votes
2answers
81 views

Estimating time in harmonic signal

I hope someone can help me with the following problem: Assume a periodic signal of the form $$\begin{align} s(t) &= \sum\limits_{p=1}^P \sin(p\Omega_0t)\\ &= \sum\limits_{p=1}^P ...
3
votes
0answers
106 views

Estimate number of distinct items

I have a large array of $n$ integers, some of which may be repeated, and I want to estimate how many distinct integers are in the array. Say the number of distinct integers is $N$. I can sample with ...
3
votes
1answer
319 views

Determine whether a statistic is sufficient, given the probability density

Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And $f_\theta=0$ if $ x ...
2
votes
3answers
83 views

More accurate estimation of mathematical constant $e$

Very often in books and also on Wikipedia we can find that: $$e \approx \left(1+\frac{1}{n}\right)^n$$ but I want more accurate estimation, it means instead using $\approx$ I wonder if I can use ...
2
votes
2answers
157 views

Is $\frac{m-1}{x}$ an unbiased estimator of $\theta$ for given pdf?

Let $X$ be a continuous random variable with pdf, $$f(x;\theta)=\frac {\theta^m.x^{m-1}e^{-\theta x}} {(m-1)!} ; x\geq0, \theta>0$$ Is $\frac{m-1}{x}$ an unbiased estimator of $\theta$ for given ...
2
votes
1answer
218 views

Parameter estimation for a distribution by minimizing its conditional entropy

Let $X$ be a discrete random variable with Laplacian distribution with mean $0$ and scale $\lambda$, as $$ p(X) = \frac{1}{2\lambda} \exp\left(-\frac{|x|}{2\lambda}\right), \\ X \in ...
2
votes
1answer
225 views

Maximum Likelihood Estimation

For iid random variables from a distribution with p.d.f. $$f(x;\theta_1,\theta_2)=\frac{1}{\theta_2}\exp\bigg(-\frac{(x-\theta_1)}{\theta_2}\bigg), \quad x>\theta_1, ...
2
votes
1answer
46 views

Monotonicity of Sample Mean

$X_1,X_2,\ldots$ are drawn i.i.d. from a distribution with mean $\mu$. Define $\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i$ Prove that $\forall t \quad E[|\bar{X}_t - \mu|] \geq E[|\bar{X}_{t+1} - \mu|]$ ...
2
votes
1answer
58 views

Finding the MLE of $\theta$ where $\theta \leq x$

consider the following PDF $$ \begin{eqnarray} f(x;\theta) &=& \left\{\begin{array}{ll} 2\frac{\theta^2}{x^3} & \theta \leqslant x\\ 0 & x< \theta; 0 < \theta ...
2
votes
2answers
85 views

Have spd $(A^TA)$ and $(B^TB)$, need $A^TB$.

Given two symmetric positive definite matrices $(A^TA)$ and $(B^TB)$ I need to compute $A^TB$. $A$ and $B$ are not given directly. $(A^TA)$ and $(B^TB)$ have the same dimensions. $A$ and $B$ are ...
2
votes
2answers
732 views

Some expectation values for a Gamma distribution

Assuming I have a Gamma distributed random Variable $x \sim Gamma( \alpha, \beta )$. Now I like to have the following two expectation values (integrals): $E \left[ x \ln x \right]$ $E \left[ \ln ...
2
votes
1answer
1k views

How do I find the MLE of $\theta$ when x is dependent on $\theta$?

Let $X_{1},X_{2},...,X_{n}$ represent a random sample from a distribution with pdf: $f(x; \theta)=e^{-(x-\theta)}, \theta \le x<\infty, -\infty<\theta<\infty$ | zero elsewhere I need to ...
2
votes
2answers
299 views

Expected value of a max

We have a roulette with the circumference $a$. We spin the roulette 10 times and we measure 10 distances, $x_1,\ldots,x_{10}$, from a predefined zero-point. We can assume that those distances are ...
2
votes
0answers
32 views

paramter estimation (maximum likelihood) of a mixture density

I have this mixture distribution $f(x) =w \cdot \mathcal{LN}(\mu_1,\sigma) + (1-w)\cdot \mathcal{LN}(\mu_2,\sigma) $ where $\mathcal{LN}(\mu,\sigma)$ is a lognormal distribution. I now have random ...
2
votes
1answer
35 views

Bayes - dual estimation of parameter value and parameter growth

I am trying to find an bayesian approach to the following problem: Image a bucket with 100 white balls and an unknown number of red balls During each year, one can take a sample with replacement of ...
2
votes
1answer
59 views

How to compute the unique MLE from an Exponential Family of Distributions?

Let $$ f(x;\theta)=\frac{1}{\pi} \frac{e^{\theta x}\cos(\theta \pi/2)}{\cosh(x)}, x\in{\mathbb{R}} $$ be a family of densities and which is clearly exponential family. Then what is the Maximum ...
2
votes
0answers
40 views

Alternatives to Fisher information

The Fisher information matrix is defined as the following: $$\mathcal{I}(\theta)=E[(\frac{\partial \log f(x;\theta)}{\partial \theta})^2]=-E[\frac{\partial^2 \log f(x;\theta)}{\partial \theta ...
2
votes
0answers
70 views

Identification of real functions

this my second question, so I'm still new... thanks in advance for any help! Basically, I'm looking for some references and tools to study the following problem. Consider the following function ...
2
votes
0answers
46 views

Unbiased estimator with conditional expectation.

Suppose that $X$ has a binomial distribution with parameter $N=1$ and $p=1/2$. Y, which is independent of $X$, has a normal distribution with mean $\mu$ and variance 1. Consider the estimator $\mu$ of ...
2
votes
1answer
22 views

How to show this estimation?

i have this polynom $$p(x) = \sum_{i=0}^{m}a_ix^i$$ I want to show, that if $\tilde{z}$ is the approximation to the simple zero digit $z \neq 0$ in first approximation, the following estimation ...
2
votes
2answers
58 views

Why we always put log() before the joint pdf when we use MLE(Maximum likelihood Estimation)?

Maybe this question is simple, but I really need some help. When we use the Maximum Likelihood Estimation(MLE) to estimate the parameters, why we always put the log() before the joint density? To use ...
2
votes
0answers
38 views

What are the properties of median-unbiased estimators?

On Wikipedia it says that " A median-unbiased estimator minimizes the risk with respect to the absolute-deviation loss function, as observed by Laplace." How to prove this? Note that I asked on Cross ...
2
votes
1answer
300 views

Maximum likelihood estimators, hypergeometric and binomial

I'm trying to solve a two part problem. The set up is as follows: consider a bag with $\theta$ red marbles and $7-\theta$ blue marbles, with $\theta$ being unknown. Let $x$ denote the number of red ...
2
votes
0answers
185 views

Fisher Information and minimum variance estimators

I am trying to understand what can be proved about minimum variance estimators. I have changed the question to make it more specific. Let us assume we have some finite set $S$ of elements and we just ...