Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

26
votes
3answers
9k views

Intuitive explanation of a definition of the Fisher information

I'm studying statistics. When I read the textbook about Fisher Information, I couldn't understand why the Fisher Information is defined like this: $$I(\theta)=E_\theta\left[-\frac{\partial^2 }{\...
11
votes
2answers
788 views

You see a route 14 bus on the moon. What is the most likely number of bus routes on the moon?

This question was asked on a forum and while many argued that the answer is 14 (since the probability of you seeing bus 14 is maximum in this case), I argued against it that they were working ...
9
votes
2answers
10k views

Difference between logarithm of an expectation value and expectation value of a logarithm

Assuming I have a always positive random variable $X$, $X \in \mathbb{R}$, $X > 0$. Then I am now interested in the difference between the following two expectation values: $E \left[ \ln X \...
9
votes
2answers
703 views

Why should Gaussian noise have fractal dimension of 1.5?

In a paper I'm trying to understand, the following time series is generated as "simulated data": $$Y(i)=\sum_{j=1}^{1000+i}Z(j) \:\:\: ; \:\:\: (i=1,2,\ldots,N)$$ where $Z(j)$ is a Gaussian noise ...
8
votes
1answer
10k views

Maximum Likelihood Estimator of parameters of multinomial distribution

Suppose that 50 measuring scales made by a machine are selected at random from the production of the machine and their lengths and widths are measured. It was found that 45 had both measurements ...
7
votes
1answer
3k views

Minimum variance unbiased estimator for scale parameter of a certain gamma distribution

Let $X_1, X_2, ..., X_n$ be a random sample from a distribution with p.d.f., $$f(x;\theta)=\theta^2xe^{-x\theta} ; 0<x<\infty, \theta>0$$ Obtain minimum variance unbiased estimator of $\theta$...
5
votes
3answers
285 views

Estimating population size

Let's suppose there are $n$ real numbers $a_0 < ... < a_n$ uniformly selected from interval [0, 1). If one knows $k$ numbers on consecutive positions $a_i < ... < a_{i+k-1}$ how good is $(...
5
votes
2answers
554 views

estimate the perimeter of the island

I'm assigned a task involving solving a problem that can be described as follows: Suppose I'm driving a car around a lake. In the lake there is an island of irregular shape. I have a GPS with me in ...
5
votes
1answer
209 views

Finding the joint distribution of $X_{1:n}$ and $\overline{X}$

I need to show that, given a random sample of independent variables $X_1, ... , X_n$, each following a distribution EXP($\theta$,$\eta$), that is, $f(x_i)=\frac{1}{\theta}\exp(-\frac{x_i-\eta}{\theta})...
4
votes
2answers
1k views

Inverse problem from pdes

A linear inverse problem is given by: $\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$ where d: observed data, A: theory operator, m: unknown model and e: error. To minimize the effect of the noise; ...
4
votes
3answers
2k views

Estimating the Gamma function to high precision efficiently?

I know there are several approximations of the Gamma function that provide decent approximations of this function. I was wondering, how can I efficiently estimate specific values of the Gamma ...
4
votes
1answer
150 views

Showing that $X_{1:n}$ is sufficient for $\eta$, by factorization

I'm asked to show that $X_{1:n}$ (the minimum order statistic) is sufficient for $\eta$, in the case of a random sample $(X_1, ... , X_n)$ where $X_i\sim EXP(1,\eta$) (this is the two-parameter ...
4
votes
1answer
308 views

Is a probability density function necessarily a $L^2$ function?

If a nonnegative continuous real valued function $f$ is integrable over $\mathbb{R}$ with $$\int_\mathbb{R} f\,\mathrm{d}x = 1,$$ does it hold true $$\int_\mathbb{R} f^2 \,\mathrm{d}x<\infty?$$ ...
4
votes
1answer
276 views

Maximum Likelihood Estimator of the exponential function parameter based on Order Statistics

Let $X_1, \ldots, X_n$ be a random sample from the exponential distribution $\exp(\lambda)$. Let $$M_n=\max\{X_1, \ldots, X_n\}$$ with probability density function $$g_{M_n}(x)=n\lambda e^{-\lambda x}(...
4
votes
2answers
141 views

Estimating the “step size” of a grid

Suppose one is given a set of $M$ points distributed on a "grid", i.e: $$x_i = x_0 + \alpha n_i + \epsilon_i, \quad n_i\in\mathbb{Z}$$ This might like something like this: $\quad\ \quad\quad\quad\...
4
votes
2answers
74 views

estimation of a parameter

The question is: $x_i = \alpha + \omega_i, $ for $i = 1, \ldots, n.$ where $\alpha$ is a non-zero constant, but unknown, parameter to be estimated, and $\omega_i$ are uncorrelated, zero_mean, ...
4
votes
1answer
76 views

MLE for the PDF $f_\theta(x)=\theta x$ on $0\leq x\leq\sqrt{2/\theta}$: where is the mistake?

Consider $f_X(x;\theta)=\theta\cdot x$, $x\leq\sqrt{\frac{2}{\theta}}$. Find the maximum likelihood for the estimator $\hat{\theta}$ of $\theta$. By definition, the MLE of $f(x_1\ldots,x_n;\hat{\...
4
votes
1answer
158 views

Estimate the scale of $e^{-(m+1) t} \sum _{k=0}^{\infty } \frac{t^k}{k!}\left(\sum _{r=0}^k \frac{t^r}{r!}\right)^{m}$

I would like to estimate the scale of the following series, $$S(m,t)=e^{-(m+1) t} \sum _{k=0}^{\infty } \frac{t^k}{k!}\left(\sum _{r=0}^k \frac{t^r}{r!}\right)^{m},$$ where $e$ is the base of ...
4
votes
2answers
3k views

How to estimate variances for Kalman filter from real sensor measurements without underestimating process noise.

As the title says, I want to estimate the variances needed for a Kalman filter from real sensor measurements only. For example we can take a temperature sensor, but the solution shall be as ...
4
votes
1answer
156 views

MLE of the mean of a heteroscedastic Gaussian time series

Suppose we observe $Y_i\sim \mathcal{N}(\theta_0 + \theta_1 x_i, \sigma_i^2)$, with $x_i$ and $\sigma_i^2$ known for all $i = 1,\ldots,n$ and $Y_1,\ldots,Y_n$ independent. Assume $\theta_0$ is ...
4
votes
0answers
59 views

Properties of an MLE based on likelihood constructed from both PDF and CDF

For continuous RV the likelihood function is (typically) given by a product of PDFs, i.e. $$L(\theta; x_1,x_2, ..., x_n) = \prod_{i=1}^n f(x_i\mid \theta) $$ However, in survival analysis with ...
4
votes
0answers
269 views

A late-diverging “approximating solution” for a system of functional equations

Peace be upon you, At the end of this question, I have shown that how computing MLE on an i.i.d Beta distributed data, results in the following system \begin{align*} &\begin{cases} \psi(\alpha)-\...
4
votes
0answers
135 views

Estimate number of distinct items

I have a large array of $n$ integers, some of which may be repeated, and I want to estimate how many distinct integers are in the array. Say the number of distinct integers is $N$. I can sample with ...
3
votes
3answers
8k views

Prove the sample variance is an unbiased estimator

I'm trying to prove that the sample variance is an unbiased estimator. I know that I need to find the expected value of the sample variance estimator $$\sum_i\frac{(M_i - \bar{M})^2}{n-1}$$ but I get ...
3
votes
2answers
157 views

Estimation of the number of prime numbers in a $b^x$ to $b^{x + 1}$ interval

This is a question I have put to myself a long time ago, although only now am I posting it. The thing is, though there is an infinity of prime numbers, they become more and more scarce the further you ...
3
votes
1answer
387 views

Parameter estimation for a distribution by minimizing its conditional entropy

Let $X$ be a discrete random variable with Laplacian distribution with mean $0$ and scale $\lambda$, as $$ p(X) = \frac{1}{2\lambda} \exp\left(-\frac{|x|}{2\lambda}\right), \\ X \in \{0,1,...,255\...
3
votes
2answers
642 views

How can I compare two Markov processes?

There is a discrete-time irreductible Markov process with $r$ possible states. $k$ observations were performed. At each observation a state of process was determined. $T_0 = \lbrace 0,1,\dots ,k-1\...
3
votes
1answer
1k views

CRLB/UMVUE estimation of $\theta$

We have a random sample $X_1,X_2,\ldots,X_n$ from a probabilitiy distribution with density $f(x;\theta) = \theta x^{-\theta-1} $ given that $x > 1$, and $0$ else. where $\theta >1 $ is an ...
3
votes
2answers
3k views

biased Maximum Likelihood estimation

Given $N$ points ($x_k$, k from $1$ to $N$) generated from a normal distribution (1-dimensional case) with known mean $\mu$, the Maximum Likelihood estimation of the variance is $\frac{1}{N}\sum_{k=1}^...
3
votes
2answers
49 views

Difficult to understand difference between the estimates on E(X) and V(X) and the estimates on variance and std.dev. on lambda-hat

I'm having a very hard time to separate estimates on population values versus estimates on sample values. I'm struggling with this exercise (not homework, self-study for my exam in introductionary ...
3
votes
1answer
93 views

Finding an efficient estimator for $\theta$ in $U[0, \theta]$ in terms of the sample maximum

This question appeared in a past exam paper, in the form: Let $X = (X_1\dotsc X_n)\in\mathbb{R}^n$ be an i.i.d. sample from $U[0, \theta], \theta>0$ Apply Rao-Blackwell's theorem to the unbiased ...
3
votes
2answers
116 views

Calculating likelihood of event based on retrospective analysis

I have a simple dataset consisting of the dates/times at which certain medications were taken by a patient. By looking retrospectively I'd like to make a best guess estimate as to which medication ...
3
votes
1answer
717 views

Maximum likelihood estimators, hypergeometric and binomial

I'm trying to solve a two part problem. The set up is as follows: consider a bag with $\theta$ red marbles and $7-\theta$ blue marbles, with $\theta$ being unknown. Let $x$ denote the number of red ...
3
votes
1answer
3k views

How do I find the MLE of $\theta$ when x is dependent on $\theta$?

Let $X_{1},X_{2},...,X_{n}$ represent a random sample from a distribution with pdf: $f(x; \theta)=e^{-(x-\theta)}, \theta \le x<\infty, -\infty<\theta<\infty$ | zero elsewhere I need to ...
3
votes
1answer
1k views

Unstable linear inverse problem: which “dampening” Tikhonov matrix should I use?

A linear inverse problem is given by: $\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$ where d: observed data, A: theory operator, m: unknown model and e: error. The Least Square Error (LSE) model ...
3
votes
1answer
269 views

Solving perturbed polynomial equations

Rather than asking the most general question possible, I will frame it in terms of what I believe is an illustrative example. Let $\epsilon>0$ be a small parameter, let $a,b>0$ and $x\in [-\...
3
votes
0answers
38 views

Conditional distribution [closed]

I am trying to figure out the derivation of Kalman filter based on Bayesian estimator. As we know, the assumption of Gauss-Markov model is used, then, the conditional distribution p(x(t)|Y(t-1))can be ...
3
votes
0answers
53 views

Trying to show convergence (in probability) of integrals using Taylor expansion

I've been working for a long time now on how to prove a proposition given in a paper about the asymptotic normality of POT-quantile estimators. Hope somebody can help me out. Proposition (i) Let $...
3
votes
1answer
78 views

How is the “cooking” done in surveys

In my country there's an official center undertaking surveys of voting intention every 4 months. However, they provide only "direct" voting intention, and the statistics obtained are usually pretty ...
3
votes
2answers
87 views

Estimating time in harmonic signal

I hope someone can help me with the following problem: Assume a periodic signal of the form $$\begin{align} s(t) &= \sum\limits_{p=1}^P \sin(p\Omega_0t)\\ &= \sum\limits_{p=1}^P \sin(\...
3
votes
1answer
56 views

Parameter optimization in probabilistic models

Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given $y$...
3
votes
1answer
464 views

Determine whether a statistic is sufficient, given the probability density

Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And $f_\theta=0$ if $ x <...
2
votes
3answers
107 views

More accurate estimation of mathematical constant $e$

Very often in books and also on Wikipedia we can find that: $$e \approx \left(1+\frac{1}{n}\right)^n$$ but I want more accurate estimation, it means instead using $\approx$ I wonder if I can use $\leq$...
2
votes
2answers
128 views

Variance of sample mean (problems with proof)

Assuming that I have $\{x_1,\ldots, x_N\}$ - an iid (independent identically distributed) sample size $N$ of observations of random variable $\xi$ with unknown mean $m_1$, variance (second central ...
2
votes
2answers
189 views

Is $\frac{m-1}{x}$ an unbiased estimator of $\theta$ for given pdf?

Let $X$ be a continuous random variable with pdf, $$f(x;\theta)=\frac {\theta^m.x^{m-1}e^{-\theta x}} {(m-1)!} ; x\geq0, \theta>0$$ Is $\frac{m-1}{x}$ an unbiased estimator of $\theta$ for given ...
2
votes
1answer
62 views

How to prove that the maximum likelihood estimator of $\theta$ is aysmptotically unbiased and cosistent

In a class we looked at this example: Let $X_1,...,X_n\sim U(0,\theta)$. Then the maximum likelihood function is $\mathcal{L}(\theta) = \begin{cases} \dfrac{1}{\theta^{n}} & \text{if } \text{...
2
votes
2answers
227 views

Convergence Rate of Sample Average Estimator

Let $X_1, X_2,\cdots$ be i.i.d. random variables with $E(X_1) = \mu, Var(X_1) = σ^2> 0$ and let $\bar{X}_n = {X_1 + X_2 + \cdots + X_n \over n}$ be the sample average estimator. Is there a way to ...
2
votes
1answer
297 views

Maximum Likelihood Estimation

For iid random variables from a distribution with p.d.f. $$f(x;\theta_1,\theta_2)=\frac{1}{\theta_2}\exp\bigg(-\frac{(x-\theta_1)}{\theta_2}\bigg), \quad x>\theta_1, \quad(\theta_1,\theta_2)\in\...
2
votes
2answers
23 views

Trying to find the MLE of $\tau$

Let $\tau = \int x \,dF(x),$ and I want to find the MLE of $\tau$ given $X_1,\ldots,X_n \sim \mathrm{Uniform}(a,b).$ I am not entirely sure, but I would imagine that $\tau = \int x \, dF(x) = \int_a^b ...
2
votes
1answer
46 views

How many data points are “enough” for linear regression?

I have data points $(x_t,y_t)$ generated from $y_t = a + b x_t + \epsilon$ where $\epsilon$ is gaussian error term with zero mean and unknown variance. I want to estimate coefficients $a$ and $b$ but ...